Skip to content. | Skip to navigation

Personal tools

Navigation

You are here: Home / Tutorial / Python Learning Paths

Python Learning Paths

狹義地說,學習 Python 的核心部份,是指認識它的關鍵字和語法,Python 的關鍵字數量大約40個,一天學三個的話,不超過兩星期就能學完,這部份算是簡單,需要持續精進的部份是,去理解各式資料結構、設計方法、函式庫、開發架構,以便找到適合自己需要的應用方式,或者說,去發掘所謂的最佳實務。

Count Objects James Powell: Python top to down, Left to right the-zen-of-python-as-related-by-masters things-every-engineer-should-know-hashmaps

5-levels-of-understanding-closures-in-python Check List Empty

語文的讀聽是為了解構,寫說才是建構,語文規模越大越需要有彈性的架構來支撐來方便協作,系統邊界在規模變大過程而脆弱化,臭蟲就容易在脆弱處產生。

寫程式的目的之一,是為了解決問題,這種情況的成果,可以視為一種公文嗎? 換個角度,在創造的場合,寫程式才容易開始具備藝術性,這比較容易讓人信服。程式語言是人類與電腦的溝通工具,轉化成機器語言時,可能只有功能性、而沒有藝術性可言,因此藝術性是對閱讀程式碼的另一群人而言,可以這樣說嗎?

純新手,推薦先閱讀前人的經驗談,看些容易消化的入門影片 #1 #2 Example

不同層次的 Python 使用者: Pythonic, Pythoneer, Pythonista programming-practices-that-make-python-more-like-an-object-oriented-programming-language take a look at the awesome new features coming in Python 3 abc: Abstract Base Classes python-data-structures

Effective Python

一個程式語言的慣用法 (idiom) 是由其使用者所定義的,Python 社群慣用 Pythonic 這個形容詞描述依循某種特定風格的程式碼,這種風格並非編譯器嚴格要求或強迫實作的,它是從使用該語言的經驗以及與其他人合作的過程中,慢慢浮現出來的。

PEP 8 Style

Naming: 使用 self 和 cls 來當作類別或類別方法的參數。

別用 if len(somelist) == 0 方式來檢查 [] 或 '' 之類的空值,使用 if not somelist 並假設空值會隱含地被估算成 False。

搞清楚 byte str unicode 的差異

Python3 有 byte str 兩種型別可用來表示字元序列,前者的 instance 含有原始的八位元值,後者的 instance 含有 Unicode 字元。Python3 的 str 與 Python2 的 unicode 用到的二進位編碼方式並沒有關聯,要將 Unicode 字元轉換成二進位資料,必須使用 encode 方法,反之要用 decode 方法。在 Python3 即使是空字串 byte 與 str 也不會相等,因此傳遞字串時,必須要特別注意字元序列的型別。

helper if appropriate

from urllib.parse import parse_qs
my_values = parse_qs('red=5&blue=0&green=', keep_blank_value=True)

print('Red: ', my_values.get('red'))
print('Green: ', my_values.get('green'))
print('Opacity: ', my_values.get('opacity'))
def get_first_int(values, key, default=0):
    found = values.get(key, [''])
    if found[0]:
        found = int(found[0])
    else:
        found = default
    return found

將複雜的運算式移到 helper function 內,特別是需要重複用到同樣邏輯的場合。

if/else 會比 or/and 這類 Boolean 運算式來得易懂。

Sequence Slicing

視覺雜訊: a[:5] vs a[0:5]

使用 somelist[-n:] 在 n 為 0 時,會得到原串列的一份拷貝。

切割後再跨步會建立出資料的 shallow copy,第一個作業應該要減少結果切片的大小,如果程式無法負擔兩個步驟的運算資源,那就考慮用 itertools 的 islice 方法,它不允許用負的 start end stride。

List Comprehension instead of map or filter

matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
flat = [x for row in matrix for x in row]
squared = [[x**2 for x in row] for row in matrix]

List Comprehension 支援多個 if 條件,而且 [x for x in a if x > 4 if x % 2 == 0] 和 [x for x in a if x > 4 and x % 2 == 0] 等效。

matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
filtered = [[x for x in row if x % 3 == 0]
           for row in matrix if sum(row) >= 10]

Comprehension 帶有兩個以上運算式過於複雜應避免。

Consider Generator instead of Huge Comprehension

it = (len(x) for x in open('/myfile.txt'))

for 或 while 緊接 else 會造成混淆,善用 try/except/else/finally。

def divide(a, b):
    try:
        return a / b
    except ZeroDivisionError as e:
        raise ValueError('Invalid Inputs') from e

函式避免回傳 None 而優先使用 Exception 讓呼叫端來處理例外。因為 if not result 的判斷式就可能導致錯誤,數值 0 None False 意義不同,但判斷式結果相同。

Readability

Readability as Usability 為何 Python 速度慢: GIL, Interpreted, Dynamic change-the-way-you-write-python-code with one-extra-character

學過其他語言的朋友,可以從高處綜觀異同處,例如 C 的語法大量被沿用,許多關鍵字跟 C 是同義,明顯的差別在於 Python 取消括號而採用縮排。Java 比較 Java Equivalent for Python Map Function Javascript Go + Python. Design Pattern: Composite and a Decorator. “Warning: Your programming career” @vardanator_pi Criteria for Useful Language

Full Stack Python 指引可能的應用方式. Python vs R

Go: use defer to clean up resources like files and locks

Abstract Base Classes

InteractivePython.org Common Trolls

Michael Driscoll: Python 101 201 wxPython Cookbook

How Does Python Handle Memory Management? Python uses private heaps to maintain its memory. So the heap holds all the Python objects and the data structures. This area is only accessible to the Python interpreter; programmers can't use it. And it's the Python memory manager that handles the Private heap. It does the required allocation of the memory for Python objects. Python employs a built-in garbage collector, which salvages all the unused memory and offloads it to the heap space. 使用 __slots__ 可減少記憶體用量

do until 

Installation

Portable Python: Python(x,y) WinPython Anaconda manage multiple versions Python Anywhere

for Windows

conda create -n tensorflow python=3.5

source active tensorflow

Basic

Python 動態型別特性來自於實作取捨,除了 C 語言的儲存值之外,它包含了關於型別的額外資訊,在結合許多物件的結構裡,這會帶來效能成本。

Dynamic Typing

Python is Call by Assignment

mutable vs immutable

Python Type Annotation Lesser Known Usages Passed by Assignment *args **kwargs formatting: __format__() Essential Python Resources Chrome Extensions IDE Comparison decorator itertools.combinations(mylist, 2) __iter__ __getitem__ decorator 可用來變造函式的行為,descriptor 是一套有特殊約定的實作規則,例如 @property 功能乃基於 descriptor 來實現。如果是寫 API 給其他工程師使用,這兩個有機會能讓 API 更簡潔易用,如果不是這類場合,只需會使用即可。

decorator explained

import sys

def usage():
    print("run_mod.py: a module demo")
    print("option: -v, show version info"

if sys.argv[1] == '-v':
    print("Version 0.1")
else:
    usage()

if __name__ == '__main__':
    print("Main Runs")

sys.path.append('../') 可以 import 相對路徑 但要避免使用 import 陷阱

import os
print(os.path.expanduser('~')

使用 _ 前置字 (例 safe_format 變成 _safe_format) 的函式模組,可以避免被外部程式呼叫。五種使用 _ 的情況 四種範例 載入 underscore 函式時可能要更改名稱 利用 sys.modules 查詢模組載入狀況

perl equiv: my $result = `grep XXX $filelist`

Learn Python in Y Minutes Python For Beginners Regular Expression

Awesome Python DeadSnakes for Debian

Simple Email Crawler String vs List of Chars

Lambda Multiple Line Expression 使用 lambda 當作 key d = {lambda x=[10]: print(x.append(11), x) : 3} Dictionary vs JSON

Python Challenge App Java: Primitive Type Functional Programming in Python Turn Functional into Comprehension Inner Function OOP #1 #2 #3

繼承的好範例 Object vs Type vs Metaclass: 繼承的 class 透過 assert 可以事先檢查

staticmethod vs classmethod

Popularity: Hackathon 除錯時常要理解 Comparison 的細節

The Dynamic Language Runtime and The Iron Languages

person groups was expected to be iterable but was None

lambda is an expression, not a statement

-        terms = filter(lambda term: term.value in self.context.groups,
+        terms = filter(lambda term: term.value in (self.context.groups or []),
                        vocabulary)

Private, Protected, Public: Python 並沒有機制阻止變數的存取或物件方法的呼叫,純粹透過社群慣例來處理這類議題,也就是 _ 和 __ 前置字

Name Mangling:

具備 DocString 的函式會成為 Public 除非明確地指定為 Private。

Recursive 範例

Descriptor: @property

Packaging Ecosystem 2016

Virtual Environment

Java Streams for Python Programmers 配合「開根號加十」成績算法但不想弄成 func 讓人看到, 可用 lambda 不易讓人發現, 但這樣不算是好理由, Java8 引入的原因可參考, 它的 Stream API 類似 Python Generator 運作原理, 不過沒有 lambda 還是可以透過 Anonymous Class 讓 Java 實現 Stream

lambda lambda-map-and-filter-in-python

Dependency Injection

Inherit vs Compose 繼承架構不該 runtime 改變

Intermediate

Binary Search vs Galloping Search

def binary_search(array, target):
    first = 0
    last = len(array) - 1
    comparisons = 0
 
    while first <= last:
        comparisons += 1
        midpoint = (first + last) // 2
        if array[midpoint] == target:
            return (midpoint, comparisons)
        elif array[midpoint] > target:
            last = midpoint - 1
        else:
            first = midpoint + 1
    return (first, comparisons)

def galloping_search(array, target):
 
    if array[0] == target:
        return (0, 1)
    comparisons = 1
 
    # Stage 1 - Find the large index
    size = len(array)
    jump = 1
    cursor = 0
    while cursor < size and array[cursor] < target:
        comparisons += 1
        jump *= 2
        cursor += jump
 
    # Stage 2 - slice the array and run the binary search
    array = array[(cursor // 2):min(cursor, size) + 1]
    sub_search = binary_search(array, target)
 
    return ((cursor // 2) + sub_search[0] - 1, sub_search[1] + comparisons)

Benefits of Strong Typing: 1) Errors / Warnings in your editor 2) Generics 3) Ability to follow a chain and figure out what type of object is required at each step 4) Refactoring
Disadvantages of Strong Type: 1) More concise and precise, Less typing 2) No badly implemented generics 3) Closures / Functions

Check List Subclass asyncore-file.dispatcher IPython Notebooks ipywidgets jupyter-js-widgets

IPython 字型設定是透過 CSS,檔案預設位於 ~/.ipython/profile_default/static/custom/custom.css,也可以透過 Theme 來設定。

http://try.github.io/

The Data Scientist Skillset

Documentation Contribution Workflow

Egg Introduction

instance method 內的 function 可以 get 到 instance variable 但不能 set

Codio: Django, Pyramid

Plone Document Check List

BananaPi + USB Camera

__future__

UnboundLocalError: local variable 'k' referenced before assignment 當匯入的資料有欄位缺漏時 產生 split() 結果無法對應 k,v 兩個值

try:
    k, v = bs(obj.getText()).find('p', {'class': 'ct'}).text.split(': ')
    if k == u'TYPE_HZ': return v # OK
except ValueError:
    pass
#if k == u'TYPE_HZ': return v # Error

Python 3.8 breaks backward compatibility with math.factorial(Decimal(int))

Advanced

Namespace by Eric Snow 三種 Namespace Container: mapping-based (VOLATILE) attribute-based (STABLE) property-based (STATIC)

obj[key]            obj.__getitem__(key)
obj[key] = value    obj.__setitem__(key, value)
del obj[key]        obj.__delitem__(key)
key in obj          obj.__contains__(key)

GIL Concurrency Issue: Grumpy Numpy integration

sub-interpreter

Double Underscore

Anti-Patterns Beautiful and Idiomatic

Google Tech Talk 2007:

Everything is Runtime: Compiletime 也是 Runtime,像 .pyc 檔案是 Cached or Borrowed Runtime 預設用來節省時間。

Execution happens in Namespaces: module, function, class 都有自己的 Namespace。

Modules are executed Top-to-Bottom: 就像 Script 方式。載入 Module 後,會從頭到尾執行內容。

def and class statements are Runtime: 執行到的時候,相關的 Function 才建立。一般的 Python 程式碼會被編譯成 Code Object 形式,執行 def 時會把這部份的 Code Object 額外包裝成 Function Object (包含參數)。

def is primarily an assignment statement: 觀察 Bytecode 的話,會發現它跟一般的 assignment 相同。Function 的內容在執行 def 時會被先編譯,但不是馬上被執行,要等到呼叫時才執行。

class statement executes suite (code block) in a separate namespace (a dict): 當 class 執行結束時,會額外建立 Class Object (包含 Namespace 內容),裡面定義的 Function (第一個參數必須是 self) 會變成 Method。

not all callables are functions, even though all functions are callables: range() is not a function call but a call to a type, pow() and sqrt() and like that are true functions, as indicated by feeding them to type().

range() 只能處理整數 numpy.arange() 可以處理浮點數

Modern Dictionary by Raymond Hettinger

How to Write Reusable Code Moving to Python3

The Tao of Python

Concurrent

Python Source Formatter

Big Data XMind

PyCharm The Good Parts Tips Managing Plone Project with PyCharm

List of List: Flat List

res = ['{} . {} = {}'.format(x, y, x * y) for x in range(10) for y in range(10)]

Generator Generator Expression vs List Comprehension Coroutines and Subroutines Tricks for System Programmers

return 代表 function 執行結束,把控制權交回,這種方式通常被稱為 Subroutine。應至少學習一個非同步架構,最好是 asyncio,未來開發不可能離開 websocket 等即時技術,http/2 也會變流行。

generics-templates-in-python/59589910#59589910

Class and Object Relation 繼承後停用物件方法 Duck Typing 3 example

String Formatting: % vs format

exit vs sys.exit

Desktop Development: #1 #2

Test Driven Development Modules with Different Versions

Stream: Why We Switch from Python to Go

32-bit platform: time.localtime()

Raspberry Pi

setting up a headless raspberry pi zero Micro SD Card dd on MacOS Tweet from Raspberry Pi with Tweepy Raspberry Pi: Full Stack Chromecast replacer 中文輸入

Excel VBA OpenOffice Calc

motionEyeOS: Video Camera

DIY Custom Linux OS with Buildroot Scientific Computing by Fernando Perez

Naming Things: Don't Use Reserved Words

$ python setup.py --classifiers
Framework :: Plone
Framework :: Plone :: 4.3
Programming Language :: Python
Programming Language :: Python :: 2.6
Programming Language :: Python :: 2.7
Topic :: Software Development :: Libraries :: Python Modules

Proxying to a Python Web Application Running in Docker

Super considered super!

Python vs C/C++ in Embedded Systems

Pythonic or Idiomatic Python

一個人必須深刻了解一種語言,才能適當使用成語來表達意思,能善用 Python 成語和禪心,就是 Pythonic

利用 type() isinstance() issubclass() 來測試 everything is object

Python2 Python3 主要差別

Python3 比 Python2 更容易寫出好品質的程式碼 Python3 Patterns, Recipes, and Idioms 移植範例: zope.app.wsgi

Never Write for loops Again Debugging List Comprehension Design Patterns

Python Worst Practice

Why Ruby Isn't Python

Drew Perttula: I think programmers do not realize which code is an idiom- the C programmer will tell you that she needed to index elements in the list, and wanted the index to start at 0 and run to the end. What's idiomatic about clearly telling the language what you want to do? C programmers probably do realize that x++ or main() {} aren't likely to look the same in python, but I'm not surprised they get mad when there's no switch/case. Python does a lot of work for you, so not only will things not always port directly from unpython to python, but they're often unnecessary once you get here.

def numbers_to_strings(argument):
  switcher = {
    0: "zero",
    1: "one",
    2: "two",
  }
  return switcher.get(argument, "nothing")

Modernised C code: use PyVarObject_HEAD_INIT

person groups was expected to be iterable but was None

Data Class in Python 3.7

useful-python-tricks

python interview questions programmers

coding tip try to code without loops ruby-on-rails-gamechanger how-to reduce memory-consumption by half-by-adding-just-one-line-of-code how-to-build-up-an-intuition-for-recursion building-a-hashing-tool-with-python https://stackoverflow.com/q/2846653 multithreading dboyliao/%E8%81%8A%E8%81%8A-python-closure The Future of Programming is Dependent Types — Programming Word of the Day

in-praise-of developers who delete code

2to3 six

https://www.youtube.com/watch?v=PlXEsrhF1iE Why Python Sucks https://medium.com/python-pandemonium/top-design-patterns-in-python-9778843d5451 https://stackoverflow.com/questions/14973963/if-the-convention-in-python-is-to-capitalize-classes-why-then-is-list-not-cap/14974045#14974045 https://www.facebook.com/100000970714286/posts/4509861589056154/ namespace package

Patterns

Factory Pattern

https://www.facebook.com/groups/pythontw/permalink/10161440515718438/ 我在 Stack Overflow 上回答過,我覺得應該是最淺白的解釋: https://stackoverflow.com/a/29163633/1592410

https://www.facebook.com/groups/pythontw/permalink/10161735265543438/ 跟重新排列 key 倒是沒什麼關係...

Python 在處理字典表達式的時候,會先創建一個空的字典物件,再依照字典表達式中的給順序添加鍵值對,該表達式可以看成是以下陳述式的依序執行:

my_dict = dict()
my_dict[True] = 'yes'
my_dict[1] = 'no'
my_dict[1.0] = 'maybe'

在 Python 中會認為上述的 Key 都是相等的,是不論在 Python 2 還是 Python 3 中,布林型別都是整數型別的子型別,幾乎在所有環境中的行為都類似於數值 0 和 1,但在轉換為字串時會分別得到 False 與 True。就 Python 而言,由於 True、1 和 1.0 都表示相同的字典鍵,因此在直譯器處理字典表達式時,會不斷地使用後續的鍵覆蓋 True 鍵的值,因此最終產生的字典只包含一個鍵。

上面這是這串原文提到的類似概念,而 Bader 在 Python Tricks 中舉出這個例子的原因,是為了介紹以下概念: - 即使 key 相等(使用 __eq__ 判別),相互之間也不會覆蓋 - 哈希值相等的 key,相互之間不會覆蓋 - 只有兩個物件相等,且哈希值相同時,dict 才會認為 key 相同 - 承上,不能夠計算哈希值的物件,不能作為 dict key