# Search Corpus Use these tools to look for specific terms and inspect their frequency distribution and the documents they occur in. The platform offers three [search modes](./search-modes.md): [Quick Search](./quick-search.md), [Basic Search](./basic-search.md), and [Advanced Search](./advanced-search.md). It is worth getting to know these before you start. Advanced Search in particular lets you use regular expressions for more complex queries, and search for named entities, corpus annotations, and longer or more specific formulations. ::::{grid} 1 1 2 2 :gutter: 2 :::{grid-item-card} Distribution of Words :link: "#distribution-of-words" :link-type: url How often your term occurs, and how that varies over time, source, and category. ::: :::{grid-item-card} Context of Words :link: "#context-of-words" :link-type: url The text surrounding each occurrence of your term. ::: :::{grid-item-card} Distribution in Documents :link: "#distribution-in-documents" :link-type: url Which documents contain your term, and where within them. ::: :::{grid-item-card} Common Pairings :link: "#common-pairings" :link-type: url Words that habitually occur alongside your term. ::: :::: ## Distribution of Words This tool has four modules. **Frequency Table** shows the most frequent forms of your search term. To see the frequency of the main form, select **Group by lemma**. Relative frequency is measured in **words per million (WPM)**, standardised per 1,000,000 words, which allows comparison across texts of different lengths. :::{dropdown} How frequent is frequent? **Very common function words** — *the, and, of, in* 50,000–100,000 WPM (5–10% of all words). *The* occurs about 60,000 times per million words. **Common content words** — *house, work, day* 1,000–5,000 WPM. **Less frequent or specialised words** — *photosynthesis, quantum, serendipity* 10–500 WPM. **Rare words** — uncommon words, technical terms, names Under 10 WPM, sometimes a single occurrence in a million words. ::: **Word Distribution over Time** shows how your term is distributed through time. You can adjust the time span by interacting with the graph. The 7-day moving average takes the average number of occurrences within seven days, smoothing out daily fluctuations to show the overall trend. **Grouped by Source** shows the distribution of your term across all sources in your corpus. **Grouped by Category** shows the distribution across source categories. Note that source categories are determined by the type of corpus. For an overview, see [Corpora](./corpora.md). ```{figure} _static/distribution.png :alt: The Distribution of Words view, showing a frequency table and a time series graph :width: 700px :name: fig-distribution Distribution of Words for the search term *Corona*. ``` :::{note} The relative frequencies shown in the diagrams are the total hits in a category divided by the total hits in the corpus. ::: :::{admonition} Learn more :class: learn A short video introduction to frequency and distribution analysis is available in [Open Educational Resources](oer-videos.md#measuring-frequency-and-distribution). ::: ## Context of Words *Context of Words* shows text segments containing your search term. Results are presented as a table: Each row shows a particular instance of the search term in the corpus. It shows five tokens to the left and right of your term by default, and you can extend this to two sentences on either side (via the menu "Refine context window size", just above the table). This view is traditional in corpus linguistics and named KWIC, short for *Key Word In Context* — and is useful when you want to see the contexts your term occurs in across the corpus. Columns can be sorted ascending or descending. :::{tip} The table also provides a link to the original source text in the 'URL' column. For journalistic corpora, this links to Swissdox Essential, a major Swiss database of journalistic data. Please ensure that you have logged into Swissdox Essential first; otherwise, the links will not work properly. For organisational corpora, the links point to the website where the text was originally published. Please note that the links only work if the website is still online, which is beyond our control. ::: ```{figure} _static/kwic.png :alt: The KWIC view, showing the search term centred with surrounding context :width: 700px :name: fig-kwic KWIC View for the search term *Corona*. ``` ## Distribution in Documents This tool helps you find documents containing a search term. Where KWIC View shows one row per occurrence, Document View shows one row per document containing your term, and adds: - **Position in Document** — each document is divided into ten equal segments along a horizontal bar, with blue squares marking approximate positions of your term. Colour intensity reflects hits in that segment relative to total hits in the document. Sorting this column lets you explore whether your terms cluster at the beginning or end of documents. - **Document Keywords** — words occurring more frequently in the document than in the corpus, adjusted for text length using Log Ratio. These provide clues about the document's topic. - **Text metadata**, including titles, which often helps in identifying relevant documents. ```{figure} _static/kwic_text.png :alt: The Document View, showing one row per document with a distribution bar :width: 700px :name: fig-document-view Document View for the search term *Corona*. ``` :::{note} *Document keywords* here differ from *corpus keywords* produced by the [Keywords](./explore-corpus.md#keywords) tool. ::: ## Common Pairings Common pairings — also called *collocations* or *co-occurrences* — are words that frequently occur together. These recurring combinations can reveal conventional and meaningful patterns of language use. By default, searching for common pairings with *Corona* finds all lemmas occurring within five words before and after it. You can widen this to ten tokens on either side. The tool displays lemmas by default, but you can switch to word forms instead. ```{figure} _static/commonpairings.png :alt: The Common Pairings view, listing collocates with association scores :width: 700px :name: fig-common-pairings Common Pairings for the search term *Migration*. ``` :::{dropdown} How common pairings are calculated Pairings are identified using the association measure **LogDice**, which asks how often two words occur together relative to how often each occurs at all. Co-occurrence means appearing within the same **context window** of *n* words before and after the search term. The measure builds on the Dice coefficient — the number of shared occurrences set against the sum of the two individual frequencies. Taking the binary logarithm and adding 14 gives the final score: ```{math} \text{logDice} = 14 + \log_2 \frac{2 \cdot f(xy)}{f(x) + f(y)} ``` **Reading the scores** The scale runs up to a theoretical maximum of 14, which would mean the two words never occur apart. Because the logarithm is base 2, every point represents a doubling: a pairing scoring 10 is twice as strongly associated as one scoring 9, and four times as strongly as one scoring 8. Scores approaching 0 indicate pairings too rare to carry interpretive weight. For example, if *Corona* occurs 10,000 times, *Pandemie* 3,000 times, and the two appear together 1,200 times within the window, the score is about 11.6 — a strong pairing. Two properties are worth keeping in mind. LogDice does not depend on the size of the corpus, so scores can be compared across corpora that differ greatly in size. It is also symmetrical: it tells you that two words keep company, but not which one attracts the other. :::