Search Pattern Cookbook

The Search function is very powerful. Searches using a RegularExpression play an important part of tapping Foswiki's full potential. Unfortunately RegularExpressions can be incredibly obscure to the uninitiated.

Most people not familiar (enough) with Regular Expressions mostly cut and paste (and maybe tweak) from existing examples. This page intends to collect lots of examples together.

Pattern 1: Extract values from a table

Problem definition

Suppose there is a topic with a table defining entries in a data form. I.e. they define select menu items in a form definition. They are then formatted like:

| *Name*  | *Type* | *Tooltip message* |
| option1 | option |                   |
| option2 | option |                   |
| option3 | option |                   |

How to extract the 'name' values, i.e. 'option1', 'option2' and 'option3' and put them in a HTML form select input?

Solution 1

The following search pattern can be employed:

<form>
    <select>
        %SEARCH{
            "^\|[^\|]*\| *option *\|"
            topic="%TOPIC%"
            type="regex"
            multiple="on"
            nonoise="on"
            format="<option>$pattern(^\| *(.*?) *\|.*)</option>"
        }%
    </select>
</form>

which is, in effect:

option1 option2 option3

Solution 2

If your table has this format:

| *Country*      |
|                |
| Afghanistan    |
| Aland Islands  |
| Albania        |
| Algeria        |
| American Samoa |
| Andorra        |

You need to skip the header row. Use this search:

<select>
    %SEARCH{
        "^\|[^\*\|]*\|"
        topic="CountryList"
        type="regex"
        multiple="on"
        nonoise="on"
        format="<option>$pattern(^\| *(.*?) *\|.*)</option>"
    }%
</select>

Which renders as: Afghanistan Aland Islands Albania Algeria American Samoa Andorra Angola Anguilla Antarctica Antigua and Barbuda Argentina Armenia Aruba Australia Austria Azerbaijan Bahamas Bahrain Bangladesh Barbados Belarus Belgium Belize Benin Bermuda Bhutan Bolivia Bosnia and Herzegovina Botswana Bouvet Island Brazil British Indian Ocean Territory British Virgin Islands Brunei Bulgaria Burkina Faso Burundi Cambodia Cameroon Canada Cape Verde Cayman Islands Central African Republic Chad Chile China Christmas Island Cocos Islands Colombia Comoros Congo - Brazzaville Congo - Kinshasa Cook Islands Costa Rica Croatia Cuba Cyprus Czech Republic Denmark Djibouti Dominica Dominican Republic East Timor Ecuador Egypt El Salvador Equatorial Guinea Eritrea Estonia Ethiopia Falkland Islands Faroe Islands Fiji Finland France French Guiana French Polynesia French Southern Territories Gabon Gambia Georgia Germany Ghana Gibraltar Greece Greenland Grenada Guadeloupe Guam Guatemala Guernsey Guinea Guinea-Bissau Guyana Haiti Heard Island and McDonald Islands Honduras Hong Kong Hungary Iceland India Indonesia Iran Iraq Ireland Isle of Man Israel Italy Ivory Coast Jamaica Japan Jersey Jordan Kazakhstan Kenya Kiribati Kuwait Kyrgyzstan Laos Latvia Lebanon Lesotho Liberia Libya Liechtenstein Lithuania Luxembourg Macao Macedonia Madagascar Malawi Malaysia Maldives Mali Malta Marshall Islands Martinique Mauritania Mauritius Mayotte Mexico Micronesia Moldova Monaco Mongolia Montenegro Montserrat Morocco Mozambique Myanmar Namibia Nauru Nepal Netherlands Netherlands Antilles New Caledonia New Zealand Nicaragua Niger Nigeria Niue Norfolk Island North Korea Northern Mariana Islands Norway Oman Pakistan Palau Palestinian Territory Panama Papua New Guinea Paraguay Peru Philippines Pitcairn Poland Portugal Puerto Rico Qatar Reunion Romania Russia Rwanda Saint Barthélemy Saint Helena Saint Kitts and Nevis Saint Lucia Saint Martin Saint Pierre and Miquelon Saint Vincent and the Grenadines Samoa San Marino Sao Tome and Principe Saudi Arabia Senegal Serbia Seychelles Singapore Slovakia Slovenia Solomon Islands Somalia South Africa South Georgia and the South Sandwich Islands South Korea Spain Sri Lanka Sudan Suriname Svalbard and Jan Mayen Swaziland Sweden Switzerland Syria Taiwan Tajikistan Tanzania Thailand Togo Tokelau Tonga Trinidad and Tobago Tunisia Turkey Turkmenistan Turks and Caicos Islands Tuvalu U.S. Virgin Islands Uganda Ukraine United Arab Emirates United Kingdom United States United States Minor Outlying Islands Uruguay Uzbekistan Vanuatu Vatican Venezuela Vietnam Wallis and Futuna Western Sahara Yemen Zambia Zimbabwe

Solution 3

If you want to grab the complete row for each occurrence of a string in a table, for instance the word "Marketing":

| a | b | c | d | e |
| Marketing | b | c | d | e|
| a | b | c | d | e |
| a | marketing | c | d | e | 
| a | b | c | d | marketing | 

use this query:

%SEARCH{
    "^\|.*?Marketing.*"
    type="regex"
    topic="%TOPIC%"
    multiple="on"
    nonoise="on"
    format="| $pattern(^\| *(.*?) *|.*)"
}%

Which renders as:

Marketing	b	c	d	e
a	marketing	c	d	e
a	b	c	d	marketing

Pattern 2: List generated from form classification

Problem

Imagine a form-based topic classification, i.e. every page has a form with several fields. How to:

1. create a search to display all topics where one form field is set to a certain value
2. create a search to filter the list above based on the values of a second form field

Test case

In practice:
Image a form with two fields:

• TopicClassification = One, Two or Three
• TopicStatus = Test or Final

We will:

1. List all topics where the TopicClassification field is set to 'Two'
2. Enable the user to filter this list based on the values of TopicStatus

Solution

%SEARCH{
    "TopicClassification='%URLPARAM{type}%'"
    type="query"
    nonoise="on"
    sort="topic"
    format="   * $topic - <font face='arial,helvetica' size='1'> 
    _last modified by_ $wikiusername _on_ $date </font> %BR% &nbsp;&nbsp;&nbsp; 
    <font face='arial,helvetica' size='1'> $formfield(TopicStatus) </font>" 
}%

The filtering select dialogue is created as in Pattern 1:

%STARTSIDEBAR%
*Filter:* %BR%
<form name="selectType" action="%SCRIPTURLPATH{"view"}%/%WEB%/" >
    <select name="type" size="1" onchange="document.location=this.value;"> 
        %SEARCH{
            "^\|[^\|]*\| *option *\|"
            topic="TopicClassification"
            type="regex" 
            nonoise="on"
            format="<option value='%BASETOPIC%?type=$pattern(^\| *(.*?) *\|.*)'>
                $pattern(^\| *(.*?) *\|.*)</option>"
        }% 
        <option value='%BASETOPIC%'>All pages</option>
    </select>
</form>
%STOPSIDEBAR% 

This will create similar functionality as Foswiki:Extensions.TopicClassificationAddOn

Pattern 3: Extract the parent of a given topic

Problem

How to get to the parent of the current topic to display on the page?

Solution 1: Using QUERY

Use the QUERY macro:

%QUERY{ "parent.name" }%

Test case

FormattedSearch

Pattern 4: Show all Children of a given topic

Problem

How to get to the list of all children of the current topic to display on the page?

Solution

The parent information is stored in the topic meta data. Do a SEARCH to find all topic parent meta data pointing to the current topic:

Children:
%SEARCH{
    "parent.name='%TOPIC%'"
    type="query"
    nonoise="on"
    format="[[$topic]]"
    separator=", "
}%

Note: Replace %TOPIC% with %BASETOPIC% if you put this SEARCH into the skin or a sidebar.

See also HierarchicalNavigation for an elaborate example.

Pattern 5: Search and display the home topics of public webs in a list

Problem

How to find and display public webs in a drop down list box.

Solution

<form>
    <select name="topic">
        <option value="%TOPIC%">Select...</option>
            %SEARCH{
                "%HOMETOPIC%"
                scope="topic"
                web="all"
                topic="%HOMETOPIC%"
                format="<option value='$web.$topic'>$web</option>"
                separator=" "
            }%
    </select>
    <input type="submit" class="foswikiSubmit" value="Go" />
</form>

Test case

Public webs can be found with the %WEBLIST% macro.

Select...

搜寻: WebHome

Main

Number of topics: 1

Sandbox

Number of topics: 1

System

Number of topics: 1

Pattern 6: Create a select box with values from a bullet list

Problem

We have a topic with a bullet list with category names. In another topic we want to offer these values in a select box dropdown.

For example, CategoryList has:

• Clients
• People
• Rooms
• Buildings

Solution

The following search pattern can be employed:

<select name="type">
    <option>Select category...</option>
    %SEARCH{
        "   *\s*.*?"
        topic="CategoryList"
        type="regex"
        multiple="on"
        casesensitive="on"
        nonoise="on"
        format="<option>$pattern(.*   \*\s*([^\n]*).*)</option>"
    }%
</select>

To render the bullet list as a comma-separated list, use the separator parameter:

%SEARCH{
    "   *\s*.*?"
    topic="CategoryList"
    type="regex"
    multiple="on"
    casesensitive="on"
    nonoise="on"
    separator=","
    format="$pattern(.*   \*\s*([^\n]*).*)"
}%

Pattern 7: Search all topics that have been moved

Problem

How would I go about listing all moved topics ?

Solution

Search for the 'moved' meta data. Type this:

Moved topics: %SEARCH{
    "moved.from=~'\w+'"
    web="all"
    type="query"
    separator=", "
    format="$web.$topic"
    nonoise="on"
}%

Test case

Moved topics:

无法搜寻，错误

Error in tempfile() using /home/www/foswiki/working/tmp/XXXXXXXXXX: Could not create temp file /home/www/foswiki/working/tmp/9BGNRvw3BS: No space left on device at /home/www/foswiki/lib/Foswiki/Sandbox.pm line 506 at /usr/share/perl5/File/Temp.pm line 1381 File::Temp::tempfile('File::Temp', 'STDERR.4842.XXXXXXXXXX', 'DIR', '/home/www/foswiki/working/tmp', 'UNLINK', 0) called at /home/www/foswiki/lib/Foswiki/Sandbox.pm line 506 Foswiki::Sandbox::sysCommand('Foswiki::Sandbox', '/bin/grep -E -l -H -- %TOKEN|U% %FILES|F%', 'TOKEN', '^%META:TOPICMOVED{.*\bfrom=\".*\w+.*\"', 'FILES', 'ARRAY(0xa94ba38)') called at /home/www/foswiki/lib/Foswiki/Store/SearchAlgorithms/Forking.pm line 109 Foswiki::Store::SearchAlgorithms::Forking::search('^%META:TOPICMOVED{.*\bfrom=\".*\w+.*\"', 'Main', 'Foswiki::Iterator::FilterIterator=HASH(0xa9b7eb0)', 'Foswiki=HASH(0x8e23088)', 'HASH(0xa9a7f30)') called at /home/www/foswiki/lib/Foswiki/Store/SearchAlgorithms/Forking.pm line 267 Foswiki::Store::SearchAlgorithms::Forking::_webQuery('Foswiki::Search::Node=HASH(0xa9b97c8)', 'Main', 'Foswiki::Iterator::FilterIterator=HASH(0xa9b7eb0)', 'Foswiki=HASH(0x8e23088)', 'HASH(0xa9a7f30)') called at /home/www/foswiki/lib/Foswiki/Store/SearchAlgorithms/Forking.pm line 182 Foswiki::Store::SearchAlgorithms::Forking::query('Foswiki::Search::Node=HASH(0xa9b97c8)', 'Foswiki::Iterator::FilterIterator=HASH(0xa9b7eb0)', 'Foswiki=HASH(0x8e23088)', 'HASH(0xa9a7f30)') called at /home/www/foswiki/lib/Foswiki/Store/VC/Store.pm line 518 Foswiki::Store::VC::Store::query('Foswiki::Store::RcsWrap=HASH(0x8e59e08)', 'Foswiki::Search::Node=HASH(0xa9b97c8)', 'Foswiki::Iterator::FilterIterator=HASH(0xa9b7eb0)', 'Foswiki=HASH(0x8e23088)', 'HASH(0xa9a7f30)') called at /home/www/foswiki/lib/Foswiki/Store/VC/Store.pm line 470 Foswiki::Store::VC::Store::searchInWebMetaData('Foswiki::Store::RcsWrap=HASH(0x8e59e08)', 'Foswiki::Search::Node=HASH(0xa9b97c8)', 'Main', 'Foswiki::Iterator::FilterIterator=HASH(0xa9b7eb0)', 'Foswiki=HASH(0x8e23088)', 'HASH(0xa9a7f30)') called at /home/www/foswiki/lib/Foswiki/Store/QueryAlgorithms/BruteForce.pm line 192 Foswiki::Store::QueryAlgorithms::BruteForce::_webQuery('Foswiki::Query::Node=HASH(0xa9ca3d8)', 'Main', undef, 'Foswiki=HASH(0x8e23088)', 'HASH(0xa8f9508)') called at /home/www/foswiki/lib/Foswiki/Store/QueryAlgorithms/BruteForce.pm line 78 Foswiki::Store::QueryAlgorithms::BruteForce::query('Foswiki::Query::Node=HASH(0xa9ca3d8)', undef, 'Foswiki=HASH(0x8e23088)', 'HASH(0xa8f9508)') called at /home/www/foswiki/lib/Foswiki/Store/VC/Store.pm line 518 Foswiki::Store::VC::Store::query('Foswiki::Store::RcsWrap=HASH(0x8e59e08)', 'Foswiki::Query::Node=HASH(0xa9ca3d8)', undef, 'Foswiki=HASH(0x8e23088)', 'HASH(0xa8f9508)') called at /home/www/foswiki/lib/Foswiki/Meta.pm line 839 Foswiki::Meta::query('Foswiki::Query::Node=HASH(0xa9ca3d8)', undef, 'HASH(0xa8f9508)') called at /home/www/foswiki/lib/Foswiki/Search.pm line 352 Foswiki::Search::searchWeb('Foswiki::Search=HASH(0xa8b0620)', 'search', 'moved.from=~\'\w+\'', 'basetopic', 'SearchPatternCookbook', '_RAW', '\x{a} "moved.from=~\'\w+\'"\x{a} web="all"\x{a} type="query"\x{a} ...', 'web', 'all', ...) called at /home/www/foswiki/lib/Foswiki/Macros/SEARCH.pm line 32 Foswiki::__ANON__() called at /home/www/foswiki/lib/CPAN/lib/Error.pm line 379 eval {...} called at /home/www/foswiki/lib/CPAN/lib/Error.pm line 371 Error::subs::try('CODE(0x98ee5c8)', 'HASH(0xa9b7950)') called at /home/www/foswiki/lib/Foswiki/Macros/SEARCH.pm line 41 Foswiki::SEARCH('Foswiki=HASH(0x8e23088)', 'Foswiki::Attrs=HASH(0xa96a480)', 'Foswiki::Meta=HASH(0x98a9918)') called at /home/www/foswiki/lib/Foswiki.pm line 3145 Foswiki::_expandMacroOnTopicRendering('Foswiki=HASH(0x8e23088)', 'SEARCH', '\x{a} "moved.from=~\'\w+\'"\x{a} web="all"\x{a} type="query"\x{a} ...', 'Foswiki::Meta=HASH(0x98a9918)') called at /home/www/foswiki/lib/Foswiki.pm line 3021 Foswiki::_processMacros('Foswiki=HASH(0x8e23088)', '---+!! Search Pattern Cookbook\x{a}\x{a}The Search function is very p...', 'CODE(0x8dcb930)', 'Foswiki::Meta=HASH(0x98a9918)', 16) called at /home/www/foswiki/lib/Foswiki.pm line 2818 Foswiki::innerExpandMacros('Foswiki=HASH(0x8e23088)', 'SCALAR(0x8dcc3b0)', 'Foswiki::Meta=HASH(0x98a9918)') called at /home/www/foswiki/lib/Foswiki.pm line 3327 Foswiki::expandMacros('Foswiki=HASH(0x8e23088)', '---+!! Search Pattern Cookbook\x{a}\x{a}The Search function is very p...', 'Foswiki::Meta=HASH(0x98a9918)') called at /home/www/foswiki/lib/Foswiki/Meta.pm line 3103 Foswiki::Meta::expandMacros('Foswiki::Meta=HASH(0x98a9918)', '---+!! Search Pattern Cookbook\x{a}\x{a}The Search function is very p...') called at /home/www/foswiki/lib/Foswiki/UI/View.pm line 412 Foswiki::UI::View::_prepare('---+!! Search Pattern Cookbook\x{a}\x{a}The Search function is very p...', 'Foswiki::Meta=HASH(0x98a9918)', 0) called at /home/www/foswiki/lib/Foswiki/UI/View.pm line 392 Foswiki::UI::View::view('Foswiki=HASH(0x8e23088)') called at /home/www/foswiki/lib/Foswiki/UI.pm line 316 Foswiki::UI::__ANON__() called at /home/www/foswiki/lib/CPAN/lib/Error.pm line 379 eval {...} called at /home/www/foswiki/lib/CPAN/lib/Error.pm line 371 Error::subs::try('CODE(0x8855958)', 'HASH(0x8e22dc8)') called at /home/www/foswiki/lib/Foswiki/UI.pm line 435 Foswiki::UI::_execute('Foswiki::Request=HASH(0x8e0ff00)', 'CODE(0x8debca0)', 'view', 1) called at /home/www/foswiki/lib/Foswiki/UI.pm line 274 Foswiki::UI::handleRequest('Foswiki::Request=HASH(0x8e0ff00)') called at /home/www/foswiki/lib/Foswiki/Engine/CGI.pm line 41 Foswiki::Engine::CGI::run('Foswiki::Engine::CGI=HASH(0x8a71ef0)') called

Pattern 8: Extract level-1 headings

How do I go about extracting the first occurance of a level-1 heading?

Solution

Use the $pattern() token. Type this:

%SEARCH{
    "^---[+][^+][^\r\n]+[\r\n]"
    type="regex"
    nonoise="on"
    header="Headings:"
    limit="5"
    format="   * [[$topic][$pattern([\r\n\-+!]+([^\r\n]*?)[\r\n].*)]]"
    footer="Found $ntopics topics with level-1 headings"
}%

Test case

Headings:

• What is an ACRONYM?
• Access Control
• A List of Wiki Administrator Documentation
• Administrator Skills Assumptions
• Admin tools

Found 5 topics with level-1 headings

---

Related Topics: UserDocumentationCategory, SearchHelp, Macros, FormattedSearch, RegularExpression