Duolingo German: On Beyond Beta

I finished the Duolingo German course! I feel like that guy in Creepshow 2, gasping "I beat you!" at the monster. And look how that turned out for him.

This guy

Like Esperanto, German was a review; I haven't tried using Duolingo to learn a language I didn't already have a background in.

Duolingo's Esperanto course is, as of this writing, still in beta, while German is a full-fledged no-Greek-alphabet-qualifiers official release. Some friends who've dabbled in Duolingo were wondering what the difference is. The Esperanto course might be in beta, but it seems to be complete - it covers all the grammar and a decent basic vocabulary. Why hasn't it moved up to the big leagues?

The most obvious difference between the German and the Esperanto courses is - well, yes, that they teach different languages. But the German course adds voice recognition. In addition to typing sentences or their translations, you're sometimes asked to repeat them or translate them aloud.

I'm genuinely curious about Duolingo's voice-recognition training. (The training for the speech recognition software, I mean, not the training for the human learners.) In many speech-to-text applications, you want to err on the side of being generous. If someone's saying their social security number, you just need to distinguish the ten digits from each other. Inputs that sound like "sero" or "vife" are, in a sense, wrong pronunciations - but they're going to be closer to "zero" and "five" than to any other legitimate answer, so you want to accept them. (This was actually done more than sixty years ago at Bell Labs.) In a language learning application, should you tolerate less variation than you would in (say) a customer service application, so students have to approximate the correct pronunciation more closely to receive credit?

But that question is purely academic. In practice, Duolingo does often give you credit for fumbled words (sometimes it's even transcribed a synonym instead of the word I actually said). At other times, though, it chokes on sentences delievered at a natural speed, but will accept them broken up into one-word chunks. (As anyone reading this knows, I think this is counterproductive – when you're learning a language, your goal is to produce sentences smoothly, not to choke out one word at a time.) Or Duolingo will totally fumble a sentence you got right, which is just maddening when it costs you your perfect score.

The Esperanto course is clearly using recordings of whole sentences (sometimes the speaker even throws in a wry intonation). When I first heard the German I thought it was using lower-quality, chopped up recordings, but I soon realized it's actually speech synthesis. That allows a wider variety of sentences to be added to a course - for example, you can easily vary subject pronouns or phrase order and multiply the acceptable sentences without having re-record them all. (And setting up speech synthesis isn't just a question of stringing sounds together; every language has rules that change how sounds are produced depending on their context, and if you don't take them into account, your speech synthesis will sound very strange indeed.)

Let this perfect score on the Esperanto progress quiz serve as my Duolingo credentials

In addition to adding the voice support, courses can't graduate from beta until the rate of reported mistakes goes down. (One of Duolingo's FAQs says the rate needs to be less than three reports per hundred users - but shouldn't there also be something about "per some specified period of time?" And whether the reports are actual errors?) When I wrote about Duolingo's Esperanto course before, I didn't complain about the Esperanto course rejecting correct answers; it is in beta, after all. And they've accepted a couple of dozen of my suggestions. I do wish it were easier to make reports - when you're doing a timed quiz, you can easily run the clock down by taking the time to report a problem - but I'm willing to sacrifice the occasional points for the good of la Afero.

As good as it gets

Finally, the non-beta courses give you a fluency score, and let you share it on LinkedIn, which no sane person will want to do. Setting aside the question of what a fluency percentage means (I can imagine what 0% fluency would be, and maybe 100%, but what's 50%?), that 46% in the picture up there appears to be the highest possible score. That's what that insecure little owl awarded me for completing the whole German skill tree and keeping it at gold.

I will happily concede that completing the skill tree doesn't prove I'm fluent. My question is: why would I ever want to show that 46% to anyone? Surely most people's response would be "Tracy claims she reads German, but Duolingo says she's only 46% fluent. That isn't even half!"1

Now, Duolingo might be more effective at teaching language skills than I initially gave it credit for. An Esperantist friend tells me that newcomers who've completed the Duolingo course have shown up at the local Esperanto society meetings able to carry on simple conversations. (And once you can do that, you can start filling in whatever gaps your initial studies left.) Duolingo gets results, and I don't argue with results.

But what I will argue with is the gamification.

You're not going to disappoint Duo today, are you?

Like many games, Duolingo lets you earn experience points and level up. The higher your level, the more experience points you need to reach the next level.

In a typical game, when you gain levels, you also gain abilities. So far, so good; you gain Duolingo levels by completing and reviewing lessons, so you're also gaining the ability to speak a language.

But in most games, those abilities also let you gain experience faster. You start off fighting hill giants, then move on to frost giants, then fire giants, and by the end you're in the Abyss fighting Lolth.

There's just no equivalent in Duolingo. You earn ten points per new lesson, and at some point you run out of new lessons. (In Esperanto, completing the skill tree put me at level 11. The larger German skill tree took me to level 17. That includes some review to keep skills at gold.) You can earn ten or twenty points for reviewing a lesson, depending on whether you use the timer. And ... that's it. Once you've completed a tree, it becomes harder to earn points, because all you can do is review.

You can also earn lingots, in-game money that you can't spend on anything. (Though apparently if you're on mobile you can buy new outfits for Duo the owl, a reward I eagerly await for PC.) But - like experience points - lingots get harder to earn as time goes on. Every completed lesson earns you two or three lingots, but after that, you only get them for leveling up (which happens less and less frequently as time passes) or for logging in a certain number of days in a row, neither of which gives you lingots at the rate you were earning them when you first started the course.

I wish I had an inspired suggestion for giving advanced Duolingo users the opportunity to gain points faster - preferably without incentivizing users to gain points in a way that doesn't actually give them language practice. And perhaps Duolingo looks different to a beginner than it does to someone brushing up on a language. I can't start studying another language at the moment, but when I do, it'll be interesting to see how well Duolingo pays off for the time I put in.

1. Now that I think about it, maybe I just know a lot of assholes (back to article)

Written on March 4, 2016