Skip to content. | Skip to navigation

Personal tools

Navigation

You are here: Home / Tips / Buildout

Buildout

Buildout 是 zc.buildout 模組的簡稱,它用來協助 Plone 專案的建置,透過設定檔的管理,開發者可以擁有一致的專案起始環境。

Buildout 提供專案環境的管理功能,開發者編輯設定檔後,能夠管理專案開發過程的必要模組。但是一般人沒辦法憑空把程式碼或設定檔寫完,需要工具或範本的協助,把專案的程式碼骨架拉出來,這就是 mr.bobZopeSkel 要做的事。

Fabric 也能處理自動化工作,建立重覆測試的環境,簡易的場合適用 Fabric,複雜的場合適合 Buildout。兩者也可以分工合作,例如 Buildout 處理模組安裝,Fabric 處理 git checkout 或 nginx restart 工作。想在 buildout.cfg 裡設定變數,可以借由 mr.scripty 協助,例子包括 Varnish Backend 切換、批次調整 Port Number、依照作業系統版本指定下載網址、確認目錄是否存在。

查詢特定模組的版本需求範例:

$ grep plone.resource  eggs/*.egg/EGG-INFO/requires.txt

Find Requirement Specs, Get All Pinnings:

$ grep -r --include=requires.txt "dependency.to.search.for" path/to/eggs-cache

local_coredev.cfg with starzel/buildout

Django with Buildout example: esdrt.buildout Pyramid Example

重新執行 bin/buildout 時,會刪除並重新建立 parts 目錄。系統運行時可以同步執行 bin/buildout 但要注意一些細節

Differences in base.cfg between the standalone and zeo setups Plone4 從開發到部署的範例

Buildout Performance Improvements: Use buildout.dumppickedversions version 0.5 or later, Use zc.buildout version 1.5 or later, Use the -N option, Clean out your egg cache and use virtualenv --no-site-packages

pip seems not support namespace packages in the same way as buildout does.

bitbucket hg repo example for source section

buildout-plone 設定值的順序會影響執行結果

避免使用下列的設定方式:

[buildout]
eggs = Products.PloneFormGen==1.7b5

在 Products.PloneFormGen 是其他模組的相依模組時,上述設定值並不會生效。

Versions

在 Buildout 2.0 之後,內建包含版本指定功能,如果設定 show-picked-versions = true 執行過程會顯示被安裝的模組版本號碼,如果所有模組事先已被指定版本號碼,執行過程就不會顯示號碼訊息,如果設定 allow-picked-versions = false 遇到沒有指定模組版本號碼的情況,就會停止執行

git remotes

zope.schema = git ${remotes:zopegit}/zope.schema.git

Extends Links

[recommand]

zc.buildout 不同版本的執行效率比較 利用 atomic commits 改善開發流程 buildout.python: Profile Guided Optimizations (PGO) increases build time significantly, but results in Python being faster at runtime.

UserWarning: Unknown distribution option: 'paster_plugins' warnings.warn(msg)
http://groups.google.com/group/pylons-discuss/browse_thread/thread/e1293d38e3068422

zc.buildout is not capable of handling setup_requires and test_requires (and hence the abuse of extra_requires for test requirements).

var 目錄檔案被 chmod 700 可能花費過長時間

Add a line to your [buildout] section:

allow-picked-versions = false

Now when you run bin/buildout and something is not pinned, buildout will complain and stop, which should give you a hint on what is not pinned. And for some background: http://maurits.vanrees.org/weblog/archive/2010/08/fake-version-pinning

處理版本衝突 Solve Version Conflict 新版本引進不被支援的語法造成 Syntax Error

Plone 4.2 zope.schema zope.interface Conflict

bootstrap.py

bootstrap.py 跟 zc.buildout 版本直接有關

Update bootstrap.py To Newest Version 移除使用 bootstrap.py 的方法

不要使用預設的 bootstrap.py 檔案 (old, busted, no longer needed)

[versions]
setuptools =
zc.buildout =
  1. First unset setuptools and zc.buildout version pins in buildout.cfg:
  2. $ virtualenv .
  3. $ ./bin/pip install zc.buildout
  4. $ ./bin/buildout

Buildout Without Internet Connection 利用 Plone UnifiedInstaller

develop.cfg 的用途,是擴充 buildout.cfg 的內容,讓開發階段的設定,更容易管理。開發階段的設定,包括安裝 plone.reload、i18ndude 之類的工具,同時 [sources] 區塊可以指定 svn 參數,把下載原始碼和安裝步驟一次搞定。如果 [sources] 裡使用 fs 參數,是 filesystem 的意思,代表要事先手動 svn co 原始碼到 src 目錄。

下列是幾個重要設定值的說明:

extends =
    buildout.cfg

代表讀完 develop.cfg 內容後,會繼續讀進 buildout.cfg 內容。

develop.cfg

更換預設埠號 http-address

$ diff -Nu develop.cfg.orig develop.cfg
--- develop.cfg.orig	2016-10-19 09:30:18.803538810 +0800
+++ develop.cfg	2016-10-19 09:33:09.122948710 +0800
@@ -65,6 +65,8 @@
 #
 # To use a python package that is being developed in your src subdirectory, use:
 # myproduct.betterplone = fs myproduct.betterplone
+# eea.facetednavigation = git git@github.com:collective/eea.factednavigation.git
+my.theme = fs my.theme
 
 
 [buildout]
@@ -117,6 +119,7 @@
 eggs +=
     Products.DocFinderTab
     plone.reload
+    my.theme
 
 parts +=
     test
@@ -124,6 +127,7 @@
     checkdocs
     mrbob
     releaser
+    i18ndude
 
 # mr.developer settings:
 always-checkout = force
@@ -131,6 +135,10 @@
 auto-checkout = *
 
 
+[instance]
+http-address = 8090
+
+
 [test]
 recipe = collective.xmltestreport
 defaults = ['--auto-color', '--auto-progress', '--ignore_dir=.git', '--ignore_dir=bower_components', '--ignore_dir=node_modules']
@@ -166,3 +174,8 @@
 eggs =
     zest.releaser
     zest.pocompile
+
+
+[i18ndude]
+recipe = zc.recipe.egg
+eggs = i18ndude
[instance]
http-address = 8090
debug-mode = on
environment-vars =
    TZ Asia/Hong_Kong
    zope_i18n_compile_mo_files true

package.name [plone3] example from collective.virtualtreecategories

$ bin/buildout -c develop.cfg
Getting distribution for 'mr.developer==1.33'.
Got mr.developer 1.33.
mr.developer: Queued 'my.pkg' for checkout.
mr.developer: Filesystem package 'my.pkg' doesn't need a checkout.
Develop: '/home/marr/Plone/zinstance/src/my.pkg'
Uninstalling zopepy.
Uninstalling instance.
Installing _mr.developer.
Generated script '/home/marr/Plone/zinstance/bin/develop'.
Installing instance.
Generated script '/opt/fresh/zinstance/bin/instance'.
Generated interpreter '/opt/fresh/zinstance/parts/instance/bin/interpreter'.
Generated script '/opt/fresh/zinstance/bin/pilconvert.py'.
Generated script '/opt/fresh/zinstance/bin/thresholder.py'.
Generated script '/opt/fresh/zinstance/bin/pildriver.py'.
Generated script '/opt/fresh/zinstance/bin/viewer.py'.
Generated script '/opt/fresh/zinstance/bin/createfontdatachunk.py'.
Generated script '/opt/fresh/zinstance/bin/pilprint.py'.
Generated script '/opt/fresh/zinstance/bin/painter.py'.
Generated script '/opt/fresh/zinstance/bin/enhancer.py'.
Generated script '/opt/fresh/zinstance/bin/pilfile.py'.
Generated script '/opt/fresh/zinstance/bin/player.py'.
Generated script '/opt/fresh/zinstance/bin/explode.py'.
Generated script '/opt/fresh/zinstance/bin/pilfont.py'.
Generated script '/opt/fresh/zinstance/bin/gifmaker.py'.
Updating repozo.
Updating backup.
Installing zopepy.
Generated interpreter '/opt/fresh/zinstance/bin/zopepy'.
Updating unifiedinstaller.
Installing test.
Generated script '/opt/fresh/zinstance/bin/test'.
Installing diazotools.
Generated script '/opt/fresh/zinstance/bin/diazocompiler'.
Generated script '/opt/fresh/zinstance/bin/diazorun'.
Generated script '/opt/fresh/zinstance/bin/diazopreprocessor'.
Installing mrbob.
Getting distribution for 'markupsafe==0.23'.
Got MarkupSafe 0.23.
Generated script '/opt/fresh/zinstance/bin/mrbob'.
Installing checkdocs.
Installing releaser.
Generated script '/opt/fresh/zinstance/bin/fullrelease'.
Generated script '/opt/fresh/zinstance/bin/postrelease'.
Generated script '/opt/fresh/zinstance/bin/lasttagdiff'.
Generated script '/opt/fresh/zinstance/bin/addchangelogentry'.
Generated script '/opt/fresh/zinstance/bin/bumpversion'.
Generated script '/opt/fresh/zinstance/bin/prerelease'.
Generated script '/opt/fresh/zinstance/bin/release'.
Generated script '/opt/fresh/zinstance/bin/longtest'.
Generated script '/opt/fresh/zinstance/bin/lasttaglog'.
Generated script '/opt/fresh/zinstance/bin/pocompile'.
Installing i18ndude.
Getting distribution for 'i18ndude==4.0.1'.
Got i18ndude 4.0.1.
Getting distribution for 'ordereddict==1.1'.
Got ordereddict 1.1.
Generated script '/opt/fresh/zinstance/bin/i18ndude'.
Versions had to be automatically picked.
The following part definition lists the versions picked:
$ bin/buildout -c develop.cfg
mr.developer: Queued 'crgis.atcontents' for checkout.
Authorization needed for 'crgis.atcontents' at 'https://subversion.assembla.com/svn/crgis/trunk/crgis.atcontents'
Username:
Password:
mr.developer: Checked out 'crgis.atcontents' with subversion.
Develop: '/home/marr/plone410/zinstance/src/crgis.atcontents'
warning: build_py: byte-compiling is disabled, skipping.
warning: install_lib: byte-compiling is disabled, skipping.
warning: easy_install: byte-compiling is disabled, skipping.

Installed /home/marr/plone410/zinstance/src/crgis.atcontents/PasteScript-1.7.3-py2.6.egg
Searching for PasteDeploy
Reading http://pypi.python.org/simple/PasteDeploy/
Reading http://pythonpaste.org/deploy/
Reading http://pythonpaste.org/deploy/paste-deploy.html
Best match: PasteDeploy 1.5.0
Downloading http://pypi.python.org/packages/source/P/PasteDeploy/PasteDeploy-1.5.0.tar.gz#md5=f1a068a0b680493b6eaff3dd7690690f
Processing PasteDeploy-1.5.0.tar.gz
Running PasteDeploy-1.5.0/setup.py -q bdist_egg --dist-dir
/home/marr/tmp/easy_install-RPkUK3/PasteDeploy-1.5.0/egg-dist-tmp-rmQ3q5 warning: build_py: byte-compiling is disabled, skipping. warning: install_lib: byte-compiling is disabled, skipping. warning: easy_install: byte-compiling is disabled, skipping. Installed /home/marr/plone410/zinstance/src/crgis.atcontents/PasteDeploy-1.5.0-py2.6.egg Searching for Paste>=1.3 Reading http://pypi.python.org/simple/Paste/ Reading http://pythonpaste.org Best match: Paste 1.7.5.1 Downloading http://pypi.python.org/packages/source/P/Paste/Paste-1.7.5.1.tar.gz#md5=7ea5fabed7dca48eb46dc613c4b6c4ed Processing Paste-1.7.5.1.tar.gz Running Paste-1.7.5.1/setup.py -q bdist_egg --dist-dir /home/marr/tmp/easy_install-lRyiEs/Paste-1.7.5.1/egg-dist-tmp-uvHkww warning: no previously-included files matching '*' found under directory 'docs/_build/_sources' warning: build_py: byte-compiling is disabled, skipping. warning: install_lib: byte-compiling is disabled, skipping. warning: easy_install: byte-compiling is disabled, skipping. Installed /home/marr/plone410/zinstance/src/crgis.atcontents/Paste-1.7.5.1-py2.6.egg warning: no previously-included files matching '*pyc' found anywhere in distribution Uninstalling zopepy. Uninstalling instance. Installing _mr.developer. Generated script '/home/marr/plone410/zinstance/bin/develop'. Installing instance. Generated script '/home/marr/plone410/zinstance/bin/instance'. Installing zopepy. Generated interpreter '/home/marr/plone410/zinstance/bin/zopepy'. Updating zopeskel. Generated script '/home/marr/plone410/zinstance/bin/paster'. Generated script '/home/marr/plone410/zinstance/bin/zopeskel'. Installing omelette. Installing i18ndude. Getting distribution for 'i18ndude'. warning: build_py: byte-compiling is disabled, skipping. warning: install_lib: byte-compiling is disabled, skipping. warning: easy_install: byte-compiling is disabled, skipping. Got i18ndude 3.2.2. Getting distribution for 'ordereddict'. warning: build_py: byte-compiling is disabled, skipping. warning: install_lib: byte-compiling is disabled, skipping. zip_safe flag not set; analyzing archive contents... warning: easy_install: byte-compiling is disabled, skipping. Got ordereddict 1.1. Generated script '/home/marr/plone410/zinstance/bin/i18ndude'. Updating backup. Updating chown. chown: Running echo Dummy references to force this to execute after referenced parts echo /home/marr/plone410/zinstance/var/backups chmod 600 .installed.cfg find /home/marr/plone410/zinstance/var -type d -exec chmod 700 {} \; chmod 744 /home/marr/plone410/zinstance/bin/* Dummy references to force this to execute after referenced parts /home/marr/plone410/zinstance/var/backups Installing test. Generated script '/home/marr/plone410/zinstance/bin/test'. Updating repozo. Updating unifiedinstaller. *************** PICKED VERSIONS **************** [versions] i18ndude = 3.2.2 #Required by: #i18ndude 3.2.2 ordereddict = 1.1 *************** /PICKED VERSIONS ***************

ConfigurationError: ('Invalid value for', 'package', 'ImportError: Module my.package has no global browser')

add __init__.py to browser/

custom zopeskel template: zopeske.unis

直接執行 Python-2.4/bin/easy-install i18ndude 是可以的。

Clock Server

FeedFeeder Update

zope-conf-additional = 
  <clock-server>
    method /mysite/@@dailyBirthdayCheck
    period 2400
    host localhost:8080
  </clock-server>

easy_install vs pip: 由於 Windows 環境的 C 函式庫常以 binary 形式安裝,pip 並不會安裝 binary 套件檔案,在 Windows 環境的支援也不夠好,不過,透過 wheel 格式的支援和普及,這項問題已被改善。在 Ubuntu 安裝 python-pip 時,會一併安裝 python-setuptools,因此 easy_install 和 pip 會併存。另外要求 setuptools ver 8+ 才能處理一些狀況。

Plone 4.3 Latest Minimal Buildout minimalplone4

templer.core branch for PasteScript

mr.bob jinja2

mr.scripty: use Python to write configuration

collective.recipe.pydevproject: generates the two files that define a PyDev Project

buildout-cache as fallback in case can not download the file

zopeskel.dexterity

install Apache MySQL PHP

System Eggs

PyPI Secured Socket Authentication and Proxy

bobtemplates.plone: jenkins.cfg vs travis.cfg vs ci.cfg travis.cfg 不再被視為好方法

Recipes

透過這個 Plugin 機制,可以為專案加入新功能,方式是從 PyPI 下載 egg 安裝,或是以 develop egg 形式進一步開發。

zest.recipe.mysql mr.scripty

plone.recipe.codeanalysis: return-status-codes = True 可搭配 git commit --no-verify 或 git commit -n 來處理 pre-commit hook

Log Rotation

Zope Instance Log Location

透過 event-log 和 access-log 的參數設定,可以指定 log 的檔案大小

pip

VirtualEnv

Inside or Outside Project

http://pypi.python.org/pypi/isotoma.depends.plone4_1

buildout.sanitycheck performs sanity checks on the buildout environment, cancelling the buildout if continuing might be unwise.

Security: 絕對不要以 root 身份執行 buildout 指令,例如 sudo ./bin/buildout 是不必要的動作。

tutorial

Larry Pitcher (unclelarry) Fake Eggs, Version Pinning, allow-picked-versions 自動安裝 git pre-commit hooks

unpicked versions

./bin/buildout -Nvvvvv |sed -ne 's/^Picked: //p' | sort | uniq  >>
unpicked-versions.cfg

warning message - buildout:

RuntimeWarning: You have iterated over the result of pkg_resources.parse_version.
This is a legacy behavior which is inconsistent with the new version class introduced in setuptools 8.0.
In most cases, conversion to a tuple is unnecessary.
For comparison of versions, sort the Version instances directly.
If you have another use case requiring the tuple, please file a bug with the setuptools project describing that need.

Dependency Gragh

簡單地說,管好 buildout 就能把專案自動化。如果再配合 http://jenkins-ci.org/ 就可以把測試到佈署的流程,也都自動化。

應用 deploy.cfg 的進階設定

Related content
Package Skeleton