Sentence nº 361350: Happy New Year!
Open data is a vital, raw material for the open web. So is language and the fact that you, as a user, have the choice to participate in the web in your own language. Think of it like this: You might get your great code or free culture project out there. But how are you going to gain users or contributors in far-off countries who don’t necessarily speak your language, if you can’t pay for professional translation services or access to proprietary, “professional” tools? Free and open language tools and higher quality language data are key for localization, and for the health of the open web. Firefox, for example, has been translated by community members into more languages than I even knew existed, an effort characterized by a high degree of precision, and with distinct human touch.
Tatoeba focuses on example sentences, not word-by-word translation
This is one of the many reasons why a projects like Tatoeba both helpful and needed from an open web perspective, and why the Mozilla Foundation wants to encourage and help the Tatoeba project by giving it a USD 2.5K Mozilla Drumbeat Grant. We hope to be able to help more in the future and to see the translation communities working closer together.
What is Tatoeba?
At the core, Tatoeba is large, free and open database of example sentences translated into several languages:
une énorme base de données de phrases traduites dans plusieurs langues .. eine große Datenbank von Beispielsätzen, die in verschiedene Sprachen übersetzt sind ..
Sentences are added and maintained collaboratively by the users. There are presently 675 000 sentences in 80+ languages with the recent addition of Klingon(!), Thai and Slovenian. In July 2010 there were ~400,000 sentences in ~50 languages, and the database is growing exponentially. Interestingly, Esperanto has shown the fastest growth rate lately – in July there were only 1000 sentences; now there are +35,000. See more impressive statistics from the Tatoeba team.
Why does the world need a free and open database of sentences? [updated]
There are more answers than I can provide here, and I encourage you to post them in the comments, as Alan Simon from Tatoeba has done – see comments
– it’s easier for amateurs to translate sentences rather than words, and thus makes building a database accessible for more people. (Alan Simon)
– Sentences are “ready-to-use” data, as opposed to single word (Alan Simon, see below)
- being free and open, the Tatoeba lends itself to building other tools (Alan Simon).
– It helps endangered languages by giving more people access to a quality learning tool, and by connecting hobbyists and rare language geeks (Alan Simon).
- Example sentences are extremely useful for learning-by-using the web. Institutions are only starting to realize the value of this new form of learning – how many kids today have not, for example, taught themselves the basics of Japanese from Manga (and indeed, Tatoeba means “for example” in Japanese).
- Tatoeba can be used for fixing and checking automated translation – including software documentation.
–- A free and open database of example sentences is helpful for localizing software and services such as Wikipedia, Creative Commons, or the Mozilla Project. It’s an important building block for the open web.
–- It offers interesting opportunities in collaboation with other free and open language projects such as the Shtooka Project which provides free audio sentences from native speakers, integrated into Tatoeba. Extremely promising too.
–- Personally, I will use Tatoeba in the future because word-by-word translation – which is offered by a myriad of services – just doesn’t cut it on its own. If, for example, you’ve translated to languages you have very little or no knowledge of, example sentences are extremely helpful to prevent the kinds of mistakes that even the best automated services make because they provide context. If like me you enjoy using Google Translate, you also know it can only do so much..
In conclusion:
Tatoeba has shown great talent and initiative within the Drumbeat Community – in spite of the Drumbeat platform’s shortcomings at an early stage – and deserves this recognition for a heroic effort in providing the raw fuel for the future language applications.You can read more about the project and become a contributor at http://tatoeba.org/ or join Tatoeba on Drumbeat. I also strongly encoure you to correct or improve my blogpost and provide more good reasons for supporting Tatoeba than I’ve provided here. Finally, if you have a spare moment, go translate Trang’s excellent “speedgeek” using universal subtitles on the above video. Merci!
![[FSF Proud Member]](images/henrikfsfbutton_aHR0cD.png){kind=small}
wow great post:)
I’m the other admin of tatoeba with Trang. In fact now it’s more something 675 000 sentences in 80+ languages (I’ve just added Klingon, Thai and Slovenian)
to add some more answers to the
Why does the world need a free and open database of sentences?
1 – because it’s easier for “non-linguist/non professional translator” aka people like me to translate sentences rather than words, and take less time than taking time to translate complete text (and to be honest I will never have the motivation to translate even a short paragraph). So it eases the process of collection textual material by making it accessible to more people.
2 – Because sentences are “ready-to-use” data, a “tatoeba” oriented example of the importance of this is, let’s imagine a new user starts to contribute in Polish, I do not speak Polish, but I see he’s making a mistake, put the frong flag, then I can do a quick search and copy paste the sentence “Proszę, zmień flagę.” (http://tatoeba.org/fre/sentences/show/611815 , I found it by searching “change flag”). This example of usage can be extended to whatever community providing you added the “common” sentences before in tatoeba
. It would have not been possible with simple dictionnary, because I would have been missing the grammar, and I would have not felt confident to use google translate for this (what if the sentence is mistranslated by google translate, as part of the admins, I can’t take the risk of being rude or so without meaning it)
3 – Helping endangered languages
I’ve fallen in love of a language, Shanghainese, tough this language is spoken by 10 millions people, it’s very hard, nearly impossible to find resources to learn it (especially if you’re looking for “free” material), and even for native, they do not have dictionaries etc. due to the Chinese government and so, no one write books in this language etc. So this language is going to disappear due to political /social pressure, because even if this language is used by some native businessman in Shanghai (so well not the typical endangered language spoken by 2 old guys in a island of the Pacific ocean) as a foreigner it’s nearly impossible to learn it, because I will not find resource from language to Shanghainese, and speaking frankly, yeah not so many foreigners are interested in learning it, and we’re not all native of the same language.
With the help of Tatoeba we can connect this language with all the others, and connecting all the people interested in it, so it’s no more “I’m alone in my room trying to gather sentences for some french-shanghainese material and maybe someone else in the world is repeating the same work for slovenian-shanghainese” .
Moreover as a consequence of point 1, if a native wants to help preserved his own language, he will be able to do it, without needing linguistic background, he only need to be able to type sentences (well I admit we need to find a solution for “oral” only languages), he even does not need to speak an other one (but there, the sentences will be waiting someone to translate them, but at least they will not be lost)
And as a consequence to point 2, then it will be easier to make people reuse it (If I’m a busy businessman, it’s easier for me to quickly search one/two easy sentences to learn by heart to please my Shanghainese customer, and even if I do not plan to learn more, by doing this, without needing to know it, I’m helping making the language still alive)
4 Help building tools
One of the problem someone is facing when he wants to create a NLP (natural language processing tool) is the lack of material (except if you’re ready to pay for it), even if he’s the best programmer of the world with the best ideas, let’s say he wants to make a software “filling the blank in the sentence” to hep people learn grammar, if he does not have data, he will not be able to accomplish it, or at least people will not use it because it’s lacking things to put in it.
So by having tatoeba, everyone can build such applicaiion, without needing to worry about “well it will take a lof of time to gather data”.
To go back to point 3, thanks to the sentences, I’m writing right now a French-Shanghainese dictionnary, though I do not pretend it will be complete or so, but at least by having this small aligned text, it’s not so difficult to figure out the word of this language and the different meaning it can have, and by doing so beeing able to build up a dictionary, or improving those already existing.
and by making it open we does not restrict what people can do with it, which permit more creativity in the reuse, for example I’ve been surprise to see that some people use the data of tatoeba for a “typing speed” game
Of course main the goal of tatoeba is to be used as a source for even better projects
What about differentiating between literary and spoken, old-fashioned and modern language?
Since when I want to translate a product using this tool the resulting translation should be consistent.
Hello Heinz! In Tatoeba one can tag the sentences. You could for example filter out sentences with the “old-fashioned” tag.