This is a joint portal of the [http://nlp.fi.muni.cz Masaryk University's NLP Centre] and [http://www.sketchengine.co.uk Lexical Computing] focusing on various tools for corpus processing.

If you have any questions or suggestions, please subscribe to the [https://groups.google.com/a/sketchengine.co.uk/forum/#!forum/noske NoSketch Engine] Google group, where you can get involved in the discussion with the developers and other users.

{{{
#!html
<table style="border-spacing: 1em"><tr>

<td class="app" style="background-color:#008000 ; background-image:url('/chrome/site/onion_nb.png')">
<a href="/wiki/Onion">
Onion (ONe Instance ONly) is a de-duplicator for large collections of texts. It can measure the similarity of paragraphs or whole documents and drop duplicate ones based on the threshold you set.

</a></td>

<td class="app" style="background-color:#800000 ; background-image:url('/chrome/site/unitok_nb.png')">
<a href="/wiki/Unitok">
Unitok is a universal text tokeniser with specific settings for many languages. It can turn plain text into a sequence of newline-separated tokens (“vertical” format), while preserving XML-like tags containing metadata.

</a></td></tr><tr>

<td class="app" style="background-color:#0080ff ; background-image:url('/chrome/site/justext_nb.png')">
<a href="/wiki/Justext">
JusText is a HTML boilerplate removal tool. It can strip navigation links, headers, footers, etc. from HTML pages and leave just regular text containing full sentences.

</a></td><td></td></tr></table>
}}}