Skip to content. | Skip to navigation

Personal tools

Navigation

You are here: Home / Tips / Search, Query, Catalog

Search, Query, Catalog

Plone 的搜尋功能主要是由 portal_catalog 工具提供,它的機制包括索引、預設顯示的資料、辭彙表,並可以應用在參照資料上,建立 reference_catalog 之類的資料,達到雙向參照的效果。

Plone 內建的搜尋功能由 plone.app.search 模組提供,可以透過 @@search?SearchableText=關鍵字@@search?Subject%3Alist=關鍵字 網址形式送出要求,搜尋結果可以進一步過濾或排序。

portal_properties/site_properties/Properties  search_enable_description_search

搭配 portal_type 來執行搜尋功能,是常見的作法,但有下列限制:1) 想要搜尋的型別必須事先寫在程式碼裡 2) 型別名稱更換名稱的話會造成困擾

Accessing the Portal Catalog Tool

明確知道 site root 名稱的時候,可以直接存取:

portal_catalog = mysite.portal_catalog

不知道 site root 名稱的時候,就要利用 getMultiAdapter 或 getToolByName 工具:

context = aq_inner(self.context)
tools = getMultiAdapter((context, self.request), name=u'plone_tools')
portal_url = tools.catalog()

還有一種方法,是利用 traverse (acquisition) 機制來找尋,但會耗費系統資源,並不建議使用:

# Use magical Zope acquisition mechanism
portal_catalog = context.portal_catalog

在 TAL 語法裡,也有類似的 traverse 方式:

<div tal:define="portal_catalog context/portal_catalog" />

使用 getToolByName 函式是最常見的方法:

from Products.CMFCore.utils import getToolByName
catalog = getToolByName(context, 'portal_catalog')

取消安全檢查,可提昇搜尋效能。

Learning By Example

利用 Python Script 就能建立客製化的查詢結果

Render Collection Result in Page

<tal:block repeat="objectItem context/collection-id/queryCatalog">

Catalog Creation Rules: ZODB 50% 容量用來儲存 Catalog

搜尋 ( 符號 Products.CMFPlone/browser/search.py

find all 'working copy' and remove them if they are old

Counting Value of an Specific Index

利用 is_default_page 過濾目錄的預設顯示項目

Clean Up Keyword Arguments From Empty Values

預設搜尋路徑 collective.rooter: Navigation Root

Sort

sort_on 數值由 Form 參數傳入 由 ControlPanel 設定

依相關程度排序 Sort by Relevance Upload Time

Cyclic Object References Couldn't Be Used

plone.app.event Import Only New Events

未指定 sort_on 參數時,預設使用 Okapi BM25 Ranking 結果,這可能導致 Title 和 SearchableText 的排序不同排序結果不穩定的討論。當 Sort on 選了 ZCTextIndex 會遇到 CatalogError 要避掉。像 Products.CMFPlone/browser/syndication/templates/search-rss.pt 原本沒事,昇級 4.3.12 後 CatalogError: Unknown sort_on index (relevance)

sorting ReferenceField values in template

Export Scripts Run From Plone

Folder Depth Condition

Sorting by Multiple Fields

List All Events in a Folder

CMFBiblographyAT not implemented search friendly

維持目錄架構 portal_user_catalog Easily change Default Search Order mockup pull request

Indexes

新增 Collection 的搜尋排序條件

The catalog has a set of methods accessing its indexes:

  • indexes() -- the list of index ids
  • index_objects() -- the list of indexes (not acquisition wrapped)
  • getIndexObjects() -- the list of indexes (acquisition wrapped)

Dexterity start field indexing tncr.policy: 從地址取出行政區作為索引值 One Index Name with Multiple Indexer

Full-text Searching

SearchableText 是 Plone 用來進行全文搜尋的索引項目,自製型別可以 override 運作方式:

from plone.indexer import indexer

@indexer(IPhoto)
def SearchableText(obj):
    return ''.join([obj.Title(), obj.Description(), obj.caption.output])

Products.EasyNewsletter/indexers.py

Dexterity 需要額外設定才能生效,collective.dexteritytextindexer 是個輔助工具,先註冊 Behavior 後,再指定 Field:

<property name="behaviors">
  <element value="collective.dexteritytextindexer.behavior.IDexterityTextIndexer" />
</property>
from collective import dexteritytextindexer

class IMyType(form.Schema):
    """My Type
    """

    dexteritytextindexer.searchable('varenummer')
    varenummer = schema.TextLine(
            title=_(u"Varenummer"),
        )

plone.directives.form 已被取消 Schema 設定範例 IRichText Behavior 可能無法被搜尋

<field name="specialfield" type="zope.schema.TextLine"
  indexer:searchable="true">
</field>

Grok 範例:

@indexer(IMyType)
def timeIndexer(obj):
    if obj.year_start:
        try:
            time = int(obj.year_start)
        except ValueError:
            time = None
    else:
        time = None
    return time
grok.global_adapter(timeIndexer, name='time_start')

Fix DexterityTextIndexer Triggered with Script

$ bin/instance run -Ozodb/path/to/Plone script.py

Plain Text Search: collective.contact.core plone.app.contenttypes 有不同的 SearchableText 索引方式 fix transform to safe_html fail

Unindexing Boolean Indexes Is Slow

And Or 運算 (可能只有 KeywordIndex 支援)

Metadata 是儲存在 Catalog 裡的資料,用來顯示搜尋結果的值。以

SearchableText 索引機制和 ElasticSearch 功能衝突?

Folderish Type 包含的 Type 也會產生索引值,利用 if obj.portal_type 判斷後,回傳 None 來改善這個結果。

Hanno Schlichting: Someone needs to look at each place where None values are indexed into the catalog and decide whether or not that was wanted or even required behavior. Some of them might need to be switched to using a different dummy value. Another approach would be to ignore any inserts of None but raise a warning instead of an exception. discussion log

Dexterity Not Indexed Nor CalalogAware

title shown to anonymous users but login required to view the detailed contents.

unrestrictedSearchResults allowedRolesAndUsers

Browser DateIndex in ZMI

在目錄裡的項目,如果要針對目錄進入索引:

@indexer(IMyType)
def category(obj):
    if obj.portal_type != 'MyType':
        return None
    else:
        return obj.aq_parent.absolute_url().split('/')[-1]

FieldIndexes by Range

item = portal_catalog({'getPrice':{'query':[2,1000],'range':'min:max'}})

時間區間的過濾方式

<input type="text"
       id="range-start"
       name="created.query:record:list:date:ignore_empty">

Date Range Filter

date_range_query = {'query': (start, end), 'range': 'min:max'}

非必填的欄位,可能因為空值,造成 TextIndexer 發生錯誤,參考 collective.nitf 範例

CompositeIndex: 包括多個 Index 屬性值,透過預先運算過的 InterSection 來減少回傳值的數量。

getCounter: support cache key generation Proof of concept: catalog caching with plone.memoize

plone.app.discussion: Collection 表格顯示方式可載入特定的 index 值

Reminder: Never register a index in catalog.xml because the index will be purged when reinstalling your package. Instead register new indexes in your setuphandlers.py.

PEP 289

for brain in catalog():
    brain.getObject().reindexObject()
objects = (brain.getObject for brain in catalog())
for obj in objects:
    obj.reindexObject()

時間與記憶體使用狀況相同

Reference Catalog

z3c.relationfield from_interfaces / to_interfaces catalog

z3c.form RelationField Dexterity Archetypes Back Reference Plone5 Back Reference

from Acquisition import aq_inner
from zope.component import getUtility
from zope.intid.interfaces import IIntIds
from zope.security import checkPermission
from zc.relation.interfaces import ICatalog

def back_references(source_object, attribute_name):
    """ Return back references from source object on specified attribute_name """
    catalog = getUtility(ICatalog)
    intids = getUtility(IIntIds)
    result = []
    for rel in catalog.findRelations(
                            dict(to_id=intids.getId(aq_inner(source_object)),
                                 from_attribute=attribute_name)
                            ):
        obj = intids.queryObject(rel.from_id)
        if obj is not None and checkPermission('zope2.View', obj):
            result.append(obj)
    return result

collective.contact.widget: queryObject returns None, getObject raise an exception if an object can't be loaded from an intid.

References from Archetypes to Dexterity

Archetypes ReferenceField Index

Bi-direction References

Reverse reference relationlist broken, circular containment error

LinguaPlone and other tools use relations, UUIDs based catalogs, to manage the linking. In the ZMI go to http:/[mysiteroot]/reference_catalog/Indexes/relationship/manage_workspace Tab "Browse" and look for the firstcolumn area "translation of" . I guess these links are there but now broken. If you do something that enforces the change of UUIDs of objects (during Import ord recreation by copying) but do not take care of the static text of the saved UUIDs in the relations, youre result is loosing all the connections. I had the same when duplicating example content that contains relations between Archteype content. As a non programmer you get nuts! Why your stuff is broken just after renaming something in the ZMI, seems weird to me, because UUIDs are there to avoid this. Doing the same in Plone is necessary better because The Plone Application cannot guess everything changed externally. Sometimes it helps to regenerate the catalogs or reinitialize LinguaPlone by invoking the view [mybaseurl]/@@language-setup-folders in the site root. Please toy around with this only in a testing copy. Never in live data! For further hints more precise description is necessary of what you are doing exactly. Especially the versions of Plone & LinguaPlone, the location and type of objects you are manipulating and every particular step of your "surgery". That also because it may behave different for e.g. Plone 3.

想把部份的查詢結果不予顯示

search sort order Default Search Order Options Mockup LiveSearch Pattern

語系選項 catalog.searchResults({'Language': ''})

UID vs UUID

UIDs are available for Archetypes content and UUIDs for both Archetypes and Dexterity since plone.app.dexterity v1.1.

plone.uuid serialization inconsistency

resolveuid wake up the object to just made a response.redirect Embedding Objects by UID

Restriction

限制 Search Portlet 只在特定目錄裡搜尋: 在 search.pt 的 <div class="LSBox"> 加進 <input type="hidden" name="path" value="/some-folder" />

The catalog automatically filters all queries (except those made using unrestrictedSearchResults) using the allowedRolesAndUsers index to only include items for which the user has the View permission. You need to add a custom indexer for that index so that all items are indexed as viewable by the Anonymous role.

Unauthorized

contentFilter={"portal_type" : ("FileAttachment",)}

Static Text Portlet 的文字預設不會被搜尋

Multiple Path Query

Products.PloneBooking 的 Python (Script) 和 Page Temple 不再能呼叫 reference_catalog 和 uid_catalog 的 getObject method,要改用 portal_catalog 來存取。plone.jsonapi.routes 在 Plone 5 無法使用 reference_catalog 工具。

Not Returning All the Objects

查詢結果與實際數量如果有差異,原因可能是:

  • 權限不足。
  • 期限失效。
  • 使用多國語系功能,只回傳現有語文的內容。

getStatusOf 回傳 None 的例子

Unrestricted Search

特定的場合裡,例如執行資料移植時,使用 results = catalog.unrestrictedSearchResults(...) 會更方便,這種方式不會檢查 effective 和 allowed_roles_and_users 的 index。

unrestrictedSearchResults vs allowedRolesAndUsers

Look for Objects with Missing Value or None as Key

AdvancedQuery

Products.AdvancedQuery 是改善 ZCatalog 搜尋功能的 Zope 模組,它相容既有的索引功能,但允許 and, or, not 運算子的任何組合,也支援 incremental filtering,還有點擊數加權的排序提示。搭配 Products.ManagableIndexdm.incrementalsearch 的話,效果會更好。它的原始碼並未透過公開方式管理

TextIndexNG hurry.query: higher level query system for the zope.catalog

search and replace in multiple document

Custom Search Forms

QUERY_STRING will be empty if a POST method is used

Semantic Search

Plone does not support by year 2010

Latent Semantic Analysis in Python by year 2008

Vector Space Search Engine Neural Network word2vec

fuzzy search

keyword weighting

Sort by Hits

collective.citationstyles: Implement Styling for Bibliography Display Types

Indexer 遇到 AttributeError 時,不會執行 index 設定,例如 ATContentTypes 使用 context.start() 和 context.end() 而不是 context.getStartDate() 和 context.getEndDate()。