Still No Templated Feature Extraction

Looking at other CRF implementations, and talking to others who’d used them, I see that designing a language to specify feature extractions is popular. Like other decisions in LingPipe, I’ve stuck to code-based solutions. The problem with this is that it limits our users to Java developers.

I normally would not make a post such as this one (my blog is not my advertisementsphere), but given that it is unlikely this paper will appear in a conference in the near future (and conferences are just ads), I decided to include a link. John, Daniel and I have been working on an algorithm called Searn for solving structured prediction problems. I believe that it will be useful to NLP people, so I hope this post deserves the small space it takes up.

A while back I came across a paper that (in a completely separate context) argues for approximating problems in lieu of approximating solutions. This idea has a nice analogue in NLP: should we (A) choose a simple model for which we can do exact inference or (B) choose a complex model that is closer to the truth for which exact inference is not tractable. (A) is approximating the problem while (B) is approximating the solution.

It seems that all signs point to (B). In almost every interesting case I know of, it helps (or at the very least doesn't hurt) to move to more complex models that are more expressive, even if this renders learning or search intractable. This story is well known in word alignment (eg, GIZA) and MT (eg, model 4 decoding), but also has simpler examples in parsing (cf, McDonald), sequence labeling (cf, Sutton), relation extraction (cf, Culotta), as well as pretty much any area in which "joint inference" has been shown to be helpful.

One sobering example here is the story in word alignment, where one cannot go and directly use, say, model 4 for computing alignments, but must first follow a strict recipe: run a few iterations of model 1, followed by a few model 2, followed by some HMM, then model 4 (skipping model 3 all together).  The problem here is that learning model 4 parameters directly falls into local minima too easily, so one must initialize intelligently, by using outputs of previous iterations.  My guess is that this result will continue to hold for training (though perhaps not predicting) with more an more complex models.  This is unfortunate, and there may be ways of coming up with learning algorithms that automatically initialize themselves by some mechanism for simplifying their own structure (seems like a fun open question, somewhat related to recent work by Smith).

Aside from a strong suggestion as to how to design models and inference procedure (i.e., ignore tractability in favor of expressiveness), there may be something interesting to say here about human language processing.  If it is indeed true that, for the most part, we can computationally move to more complex models, forgoing tractable search, then it is not implausible to imagine that perhaps humans do the same thing.  My knowledge in this area is sparse, but my general understanding is that various models of human language processing are disfavored because they would be too computationally difficult.  But if, as in old-school AI, we believe that humans just have a really good innate search algorithm, then this observation might lead us to believe that we have, ourselves, very complex, intractable "models" in our heads.

In an ideal world, it would be possible to read a paper, go out and implement the proposed algorithm, and obtain the same results. In the real world, this isn't possible. For one, if by "paper" we mean "conference paper," there's often just not enough space to spell out all the details. Even how you do tokenization can make a big difference! It seems reasonable that there should be sufficient detail in a journal paper to achieve essentially the same results, since there's (at least officially) not a space issue. On the other hand, no one really publishes in journals in our subfamily of CS.

The next thing one can do is to release the software associated with a paper. I've tried to do this in a handful of cases, but it can be a non-trivial exercise. There are a few problems. First, there's the question of how polished the software you put out should be. Probably my most polished is megam (for learning classifiers) and the least polished is DPsearch (code from my AI stats paper). It was a very nontrivial amount of effort to write up all the docs for megam and so on. As a result, I hope that people can use it. I have less hope for DPsearch --- you'd really have to know what you're doing to rip the guts out of it.

Nevertheless, I have occasionally received copies of code like my DPsearch from other people (i.e., unpolished code) and have still been able to use them successfully, albeit only for ML stuff, not for NLP stuff. ML stuff is nice because, for the most part, its self-contained. NLP stuff often isn't: first you run a parser, then you have to have wordnet installed, then you have to have 100MB of data files, then you have to run scripts X, Y and Z before you can finally run the program. The work I did for my thesis is a perfect example of this: instead of building all the important features into the main body of code I wrote, about half of them were implemented as Perl scripts that would essentially add "columns" to a CoNLL-style input format. At the end, the input was like 25-30 columns wide, and if any were missing or out of order, bad things would happen. As a result, it's a completely nontrivial exercise for me to release this beast. The only real conceivable option would be to remove the non-important scripts, get the important ones back into the real code, and then release that. But then there's no way the results would match exactly those from the paper/thesis.

I don't know of a solution to this problem. I suppose it depends on what your goal is. One goal is just to figure out some implementation details so that you can use them yourself. For this, it would be perfectly acceptable in, say, my thesis situation, to just put up the code (perhaps the scripts too) and leave it at that. There would be an implicit contract that you couldn't really expect too much from it (i.e., you shouldn't expect to run it).

A second goal is to use someone else's code as a baseline system to compare against. This goal is lessened when common data is available, because you can compare to published results. But often you don't care about the common data and really want to see how it works on other data. Or you want to qualitatively compare your output to a baseline. This seems harder to deal with. If code goes up to solve this problem, it needs to be runnable. And it needs to achieve pretty much the same results as published, otherwise funny things happen ("so and so reported scores of X but we were only able to achieve Y using their code", where Y

See here for the current list. They include: Nonparametric Bayes (woohoo!), machine learning and music, Bayesian modeling applications, prior knowledge for text and language processing, sparse optimization and variable selection, as well as stand-alone workshops on the reinforcement learning competition and mining and learning with graphs.

Because I'm one of the organizers, I'd like to call attention to the Prior knowledge for text and language processing workshop. We'd definitely like submissions on any of the following topics:

• Prior knowledge for language modeling, parsing, translation
• Topic modeling for document analysis and retrieval
• Parametric and non-parametric Bayesian models in NLP
• Graphical models embodying structural knowledge of texts
• Complex features/kernels that incorporate linguistic knowledge; kernels built from generative models
• Limitations of purely data-driven learning techniques for text and language applications; performance gains due to incorporation of prior knowledge
• Typology of different forms of prior knowledge for NLP (knowledge embodied in generative Bayesian models, in MDL models, in ILP/logical models, in linguistic features, in representational frameworks, in grammatical rules…)
• Formal principles for combining rule-based and data-based approaches to NLP
• Linguistic science and cognitive models as sources of prior knowledge

Yes, I know that's a shameless plug, but do you really expect better from me?!

I remember a few years ago complaining about automatic evaluation at conference was the thing to do. (Ironically, so was writing papers about automatic evaluation!) Things are saner now on both sides. While what I'm writing here is interpretable as a gripe, it's really intended as a "did anyone else notice this" because it's somewhat subtle.

The evaluation metric I care about is Rouge, designed for summarization. The primary difference between Rouge and Bleu is that Rouge is recall-oriented while Bleu is precision-oriented. The way Rouge works is as follows. Pick an ngram size. Get a single system summary H and a single reference summary R (we'll get to multiple references shortly). Let |H| denote the size of bag the defined by H and let |H^R| denote the bag intersection. Namely, the number of times some n-gram is allowed to appear in H^R is the min of the number of times it appears in H and R. Take this number and divide by |R|. This is the ngram recall for our system on this one example.

To extend this to more than one summary, we simple average the Rouges at each individual summary.

Now, suppose we have multiple references, R_1, R_2, ..., R_K. In the original Rouge papers and implementation, we compute the score for a single sentence as the max over the references of the Rouge on that individual reference. In other words, our score is the score against a single reference, where that reference is chosen optimistically.

In later Rouge paper and implementation, this changed. In the single-reference case, our score was |H^R|/|R|. In the multiple reference setting, it is |H^(R_1 + R_2 + ... + R_K)|/|R_1 + R_2 + ... + R_K|, where + denotes bag union. Apparently this makes the evaluation more stable.

(As an aside, there is no notion of a "too long" penalty because all system output is capped at some fixed length, eg., 100 words.)

Enough about how Rouge works. Let's talk about how my DUC summarization system worked back in 2006. First, we run BayeSum to get a score for each sentence. Then, based on the score and about 10 other features, we perform sentence extraction, optimized against Rouge. Many of these features are simple patterns; the most interesting (for this post) is my "MMR-like" feature.

MMR (Maximal Marginal Relevance) is a now standard technique in summarization that aims to allow your sentence extractor to extract sentences that aren't wholly redundant. The way it works is as follows. We score each sentence. We pick as our first sentence the sentence with the highest score. We the rescore each sentence to a weighted linear combination of the original score and minus the similarity between the proposed second sentence and its similarity to the first. Essentially, we want to punish redundancy, weighted by some parameter a.

This parameter is something that I tune in max-Rouge training. What I found was that at the end of the day, the value of a that is found by the system is always negative, which means that instead of disfavoring redundancy, we're actually favoring it. I always took this as a notion that human summaries really aren't that diverse.

The take-home message is that if you can opportunistically pick one good sentence to go in your summary, the remaining sentences you choose should be as similar to that one was possible. It's sort of an exploitation (not exploration) issue.

The problem is that I don't think this is true. I think it's an artifact, and probably a pretty bad one, of the "new" version of Rouge with multiple references. In particular, suppose I opportunistically choose one good sentence. It will match a bunch of ngrams in, say, reference 1. Now, suppose as my second sentence I choose something that is actually diverse. Sure, maybe it matches something diverse in one of the references. But maybe not. Suppose instead that I pick (roughly) the same sentence that I chose for sentence 1. It won't re-match against ngrams from reference 1, but if it's really an important sentence, it will match the equivalent sentence in reference 2. And so on.

So this is all nice, but does it happen? It seems so. Below, I've taken all of the systems from DUC 2006 and plotted (on X) their human-graded Non-Redundancy scores (higher means less redundant) against (on Y) their Rouge-2 scores.



Here, we clearly see (though there aren't even many data points) that high non-redundacy means low Rouge-2. Below is Rouge-SU4, which is another version of the metric:

Again, we see the same trend. If you want high Rouge scores, you had better be redundant.

The point here is not to gripe about the metric, but to point out something that people may not be aware of. I certainly wasn't until I actually started looking at what my system was learning. Perhaps this is something that deserves some attention.

I've been thinking a lot recently about how to do MCMC on massively parallel architectures, for instance in a (massively) multi-core setup (either with or without shared memory).

There are several ways to approach this problem.

The first is the "brain dead" approach. If you have N-many cores, just run N-many parallel (independent) samplers. Done. The problem here is that if N is really like (say 1000 or greater), then this is probably a complete waste of space/time.

The next approach works if you're doing something like (uncollapsed) Gibbs sampling. Here, the Markov blankets usually separate in a fairly reasonable way. So you can literally distribute the work precisely as specified by the Markov blankets. With a bit of engineering, you can probably do this is a pretty effective manner. The problem, of course, is if you have strongly overlapping Markov blankets. (I.e., if you don't have good separation in the network.) This can happen either due to model structure, or due to collapsing certain variables. In this case, this approach just doesn't work at all.

The third approach---and the only one that really seems plausible---would be to construct sampling schemes exclusively for a massively parallel architecture. For instance, if you can divide your variables in some reasonable way, you can probably run semi-independent samplers that communicate on an as-needed basis. The form of this communication might, for instance, look something like an MH-step, or perhaps something more complex.

At any rate, I've done a bit of a literature survey to find examples of systems that work like this, but have turned up surprisingly little. I can't imagine that there's that little work on this problem, though.

Welcome to everyone to ACL! It's pretty rare for me to end up conferencing in a country I've been before, largely because I try to avoid it. When I was here last time, I stayed with Yee Whye, who was here at the time as a postdoc at NUS, and lived here previously in his youth. As a result, he was an excellent "tour guide." With his help, here's a list of mostly food related stuff that you should definitely try while here (see also the ACL blog):

• Pepper crab. The easiest to find are the "No Signboard" restaurant chain. Don't wear a nice shirt unless you plan on doing laundry.
• Chicken rice. This sounds lame. Sure, chicken is kind of tasty. Rice is kind of tasty. But the key is that the rice is cooked in or with melted chicken fat. It's probably the most amazingly simple and delicious dish I've ever had. "Yet Kun" (or something like that) is along Purvis street.
• Especially for dessert, there's Ah Chew, a Chinese place around Liang Seah street in the Bugis area (lots of other stuff there too).
• Hotpot is another local specialty: there is very good spicy Szechuan hotpot around Liang Seah street.
• For real Chinese tea, here. (Funny aside: when I did this, they first asked "have you had tea before?" Clearly the meaning is "have you had real Chinese tea prepared traditionally and tasted akin to a wine tasting?" But I don't think I would ever ask someone "have you had wine before?" But I also can't really think of a better way to ask this!)
• Good late night snacks can be found at Prata stalls (eg., indian roti with curry).
• The food court at Vivocity, despite being a food court, is very good. You should have some hand-pressed sugar cane juice -- very sweet, but very tasty (goes well with some spicy hotpot).
• Chinatown has good Chinese dessert (eg., bean stuff) and frog leg porridge.

Okay, so this list is all food. But frankly, what else are you going to do here? Go to malls? :). There's definitely nice architecture to be seen; I would recommend the Mosque off of Arab street; of course you have to go to the Esplanade (the durian-looking building); etc. You can see a few photos from my last trip here.

Now, I realize that most of the above list is not particularly friendly to my happy cow friends. Here's a list of restaurants that happy cow provides. There are quite a few vegetarian options, probably partially because of the large Muslim population here. There aren't as many vegan places, but certainly enough. For the vegan minded, there is a good blog about being vegan in Singapore (first post is about a recent local talk by Campbell, the author of The China Study, which I recommend everyone at least reads). I can't vouch for the quality of these places, but here's a short list drawn from Living Vegan:

• Mushroom hotpot at Ling Zhi
• Fried fake meat udon noodles (though frankly I'm not a big fan of fake meat)
• Green Pasture cafe; looks like you probably have to be a bit careful here in terms of what you order
• Yes Natural; seems like it has a bunch of good options
• Lotus Veg restaurant, seems to have a bunch of dim sum (see here, too)
• If you must, there's pizza
• And oh-my-gosh, there's actually veggie chicken rice, though it doesn't seem like it holds up to the same standards as real chicken rice (if it did, that would be impressive)

Okay, you can find more for yourself if you go through their links :).

Enjoy your time here!

Quick update: Totally forgot about coffee.

If you need your espresso kick, Highlander coffee (49 Kampong Bahru Road) comes the most recommended, but is a bit of a hike from the conference area. Of course, you could also try the local specialty: burnt crap with condensed milk (lots and lots of discussion especially on page two here).

I remember when I took my first "real" Syntax class, where by "real" I mean "Chomskyan." It was at USC in Fall 2001, taught by Roumyana Pancheva. It was hard as hell but I loved it. However, as a computationally minded guy, I remember snickering to myself the whole time we were talking about movements that get you from deep structure to surface structure. This stuff was all computationally ridiculous.

But why was it computationally ridiculous? It was ridiculous because my mindset, and I think the mindset of most computational folks at the time, was that of n^3 CKY or Earley style parsing. Namely exact parsing in a context free manner. This whole idea of transformations would kill anything like that in a very bad way.

However, there's been a recent shift in attitudes. Sure, people still do their n^3 parsing, but of course none of it is exact anyway (due to pruning). But more than that, things like linear time parsing algorithms as popularized by people like Joakim Nivre and Kenji Sagae and Brian Roark and Joseph Turian, have proved very useful. They work well, are incredibly efficient, and are easy to implement. They're also a bit more psychologically plausible (as Eugene Charniak said recently "we don't know what people are doing, but they're definitely not doing CKY.").

So I'm led to wonder: could we actually do parsing in a transformational grammar using all the new stuff we know about (for instance) left-to-right parsing?

One thing that stands in our way, of course, is the stupid Penn Treebank, which was annotated only with very simple transformations (mostly noun phrase movements) and not really "deep" transformations as most Chomskyan linguists would recognize them.

But I think you could still do it. It would end up as being partially unsupervised, but at least from a minimum description length perspective, I can either spend weights learning more special cases, or I can learn general transformational rules. It would take some thought and effort to write it out and figure out how to actually optimize such a thing, but I bet it could be done in a semester.

So then the question is: aside from smaller models (potentially), is there any other reason to do it?

I can think of at least one: parsing non-declarative sentences. Since almost all sentences in the Treebank are declarative, parsers do pretty crappy when tested on other things. Slav Petrov had a paper at EMNLP 2010 on parsing questions. Here is the abstract, which says pretty much everything:

... We show that dependency parsers have more difficulty parsing questions than constituency parsers. In particular, deterministic shift-reduce dependency parsers ... drop to 60% labeled accuracy on a question test set. We propose an uptraining procedure in which a deterministic parser is trained on the output of a more accurate, but slower, latent variable constituency parser (converted to dependencies). Uptraining with 100K unlabeled questions achieves results comparable to having 2K labeled questions for training. With 100K unlabeled and 2K labeled questions, uptraining is able to improve parsing accuracy to 84%, closing the gap between in-domain and out-of-domain performance.

Now, at least in principle, if you can parse declarative sentences, you should be able to parse questions. At least if you know about some basic syntactic transformations in English. (As an aside, the "uptraining" idea is almost exactly the same as the structure compilation idea that Percy, Dan and I had at ICML 2008, though Slav and colleagues apply it to a domain adaptation problem, while we just did simple semi-supervised learning.)

We have observed similar effects in the parsing of commands, such as "Put your head in a noose" where parsers -- even constituency ones -- really really want "Put" to be a noun! Again, if you know simple transformations -- like subject dropping -- you should be able to parse commands if you can already parse declarations.

As with any generalization, the hope is that by realizing the generalization, you don't need to store so many specific cases. So if you can learn that commands and questions are simple transformation on declarative sentences, and you can learn to parse declaratives, you should be able to handle the other case.

(Anticipating comments: yes, I know you could try to pre-transform your data, like they do in MT, but that's quite inelegant. And yes, I know you could probably take the treebank and turn a lot of the sentences into commands or questions to create a new data set. But that's kind of missing the point: I don't want to just handle commands or questions... I want to handle anything, even things that I might not have anticipated.)

Bob Carpenter has blogged about a new Intro to IR book online here. I'm looking forward to skimming it this weekend. I would also recommend the Python based NLTK Toolkit.

Books and resources like these are generally geared towards people with existing programming background. If a linguist with no programming skills is interested in learning some computational linguistics, Mike Hammond has written a couple of novice's intro books called Programming For Linguists. A novice would be wise to start with Hammond's books, move to the NLTK tutorials, then move on to a more serious book like Manning et al.

And if you're at all curious about what a linguist might DO once she has worked through all that wonderful material, you might could go to my own most wonderful List of Companies That Hire Computational Linguists page here.

And if you're not challenged by any of that above, I dare you to read Bob's Type-Logical Semantics. Go on, you think yer all smart and such. I dare ya! I read it the summer of 1999 with a semanticist, a logician, and a computer scientist and it made all of our heads hurt. I still have Chapter 10 nightmares.

I may have misstated Chatham’s belief’s below. It’s not clear that he agrees with the claim I complained about. But, his blog makes it clear that he believes this:

experience can be coded in a non-linguistic form, and that recoding into language is possible, at least over short delays

First, I didn’t realize it was at all controversial that experience can be coded in non-linguistic form. Of course it can. Does anyone doubt this? Second, I have no clue what Chatham means by recoding into language. Certainly thoughts and memories can be expressed by language, that should go without saying; but, Chatham seems to believe that at least some thoughts and memories are STORED in language form. This sounds like the old “we think in language” argument.

I am not convinced that we think in language. In fact, I seriously doubt we think in language. I think language is always a post-thought process.

Tigrigna -- A language of Ethiopia

Population -- 3,224,875 in Ethiopia (1998 census). 2,819,755 monolinguals.
Region Tigray Province. Also spoken in Eritrea, Germany, Israel.

Alternate names -- Tigrinya, Tigray
Classification -- Afro-Asiatic, Semitic, South, Ethiopian, North
Language use -- National language. 146,933 second-language speakers.
Language development -- Literacy rate in first language: 1% to 10%.
Literacy rate in second language: 26.5%. Ethiopic script. Radio programs. Grammar. Bible: 1956. Comments -- Speakers are called 'Tigrai'. (screen grab from Psycholinguistics Arena)

What the hell happened on May 13, 2008?
(screen shot of a guest at McCain's BBQ. Video here)
Political pundits almost pathologically believe they have greater influence than they really do. Case in point, Talking Points Memo's editor and publisher and chief blogger Josh Marshal has been trying to promote the use of the phrase "ride the swing" as a metaphor for the case when "a reporter who has gotten way too cozy with a politician and has had their supposed objectivity affected" (original explanation here). The phrase refers to a posh BBQ that McCain hosted at one of his Arizona ranches where journalists were treated to a very comfy social experience that bordered on bribery (click on "video here" below the pic). As far as I can tell, Marshal is the primary pusher of the phrase and its most frequent user (a couple other examples here and here).
I suspect Marshal's linguistic campaign will fail. Attempts by a single person to explicitly promote the use of a new metaphor are rarely successful. This is not how language works. Successful new coinages are generally adopted less self-consciously. The process is not well understood, but examples like Marshal's are few and far between. Additionally, there are already several good metaphors for related frames, such as "drank the cool-aid" (which has equally obscure origins involving jungles and religious cults). Not sure we need a new one just for journalists.
(HT to my colleague CC for bringing this to my attention. At first, we had no clue what this metaphor referred to, and as such we literally couldn't understand what it was meant to evoke. CC did some blogger detective work and discovered its origin).
(screen shot from MSNBC's video)

Having nothing whatever to do with linguistics, nonetheless I feel compelled to report what seems like an entirely unreported snub by US President Barack Obama to the President of Argentina Cristina Kirchner. Watch MSNBC's video of the second photo shoot and you'll see Obama walk across the entire group to go shake hands with Canada's PM Stephen Harper (who missed the original shoot), but he passed right in front of Kirchner who raised her hand out to shake Obama's, but he ignored her entirely (creating a somewhat awkward moment), shook Harper's hand, then refused to make eye contact with Kirchner afterwords. I count that as two snubs.

Watch the video at Olbermann's "Countdown" site and at about 40 seconds in you'll see the moments I'm talking about. MSNBC's footage seems to be the only one with a wide enough angle to show the snubs.

The relevant frootage is here:
April 2, 2009; #5 "Obama meets the world press" (psssst, this has nothing to do with anything; just random rumor mongering...which is fun, ya know...)
(screen shot of this blog's Sitemeter data)

A reader apparently was interested in "verbs that describes tacos." Since the IP address shows the Indiana Department of Education, I got 20 bucks says this was done by a lunch lady writing out next week's menu.

As for the "linguistics aspect", well, verbs don't describe nouns (like "tacos"), adjectives do. Verbs represent events. Rather, adjectives describe nouns. So, in the interest of serving my readers, exactly what kind of of adjectives describe tacos? Let's go to the experts:

Taco Bell:

• crunchy taco
• soft taco
• taco supreme(bonus points for postnominal adjective)
• double decker taco.

This is a response to Liberman's Saturday morning goofiness here: Nice one! A web-based dictionary you can search with regular expressions (HT MetaFilter). from the site's introduction page:

The Regex Dictionary is a searchable online dictionary, based on The American Heritage Dictionary of the English Language, 4th edition, that returns matches based on strings —defined here as a series of characters and metacharacters— rather than on whole words, while optionally grouping results by their part of speech. For example, a search for "cat" will return any words that include the string "cat", optionally grouped according to gramatical category:

* Adjectives: catastrophic, delicate, eye-catching, etc.
* Adverbs: marcato, staccato, etc.
* Nouns: scat, category, vacation, etc.
* Verbs: cater, complicate, etc.

In other words, the Regex Dictionary searches for words based on how they are spelled; it can find:

* adjectives ending in ly (197; ex.: homely)
* words ending in the suffix ship (89)
o Adjectives (1, midship)
o Nouns (80; ex.: membership)
o Suffixes (1, -ship)
o Verbs (6; ex.: worship)
* words, not counting proper nouns, that have six consecutive consonants, including y (79; ex.: strychnine)
* words, not counting proper nouns, that have six consecutive consonants, not counting y (2; ex.: latchstring)
* words of 12 or more letters that consist entirely of alternate consonants and vowels (45; ex.: legitimatize)
("please, wait a moment", image from braille.ch)

Vivian Aldridge has a nice website devoted to explaining braille systems for different languages (HT Boing Boing). If I understand correctly, tone is rarely represented for Chinese braille (let's forgive for the moment that "Chinese" is the name of a language family, not a particular language):

In the few examples of Chinese braille that I have come across, the signs for the tones were not used except in the following cases:

• with the syllable yi, for which a good Chinese-German dictionary lists almost 50 different inkprint characters. In this case the indication of the tone helps to limit the number of possible meanings.
• in words where a syllable with a suppressed vowel comes before syllable without a consonant, for example the word sh'yong (try out, test) in which the braille sign for the fourth tone is used instead of the apostrope. In this case the tone sign seems to be used to separate the two syllables.

Tone is a non-trivial feature of Chinese languages. Omniglot has a nice page with the system displayed (fyi, it cites braille.ch as one of its sources). The interesting point is that tone has the ability to be represented, but according to Vivian, it normally is not (however, she notes that she has only seen a few examples). I spent two years in college struggling in Mandarin courses. I would have liked to have dispensed with tone. I was stupid enough to click through to Huffington Post's colossally stupid and fundamentally mistaken Worst Grammar Mistakes Ever post (I refuse to link to it). Of course, the 11 items had virtually nothing to do with grammar (the vast majority were punctuation and spelling errors). I must agree with Zwicky's pessimism regarding National Grammar Day: "It seems to me that the day is especially unlikely to provide a receptive audience for what linguists have to say."

But what prompted this post was the ad at the bottom for Grammarly, a free online "proofreader and grammar coach" which promised to Correct All Grammar Errors And Plagiarism.

A bold claim, indeed. I doubt a team of ten trained linguists could could felicitously make this claim. But the boldness does not stop there (it never does on the innerwebz). Click through to the online tool and wow, the bold claims just start stacking up like flap jacks at a Sunday fundraiser.

Just paste in your test and bam! you get
150+ Grammar Checks
Get detailed error explanations.
Plagiarism Detection
Find unoriginal text.
Text Enhancement
Use better words.
Contextual Spell Check
Spot misused words.

Dang! Them fancy computers, they sure is smart. Just for funnin, I pasted the text of Zwicky's NGD again post into the window and ran the check. Here's his report:

Not bad for a professor at one of the lesser linguistics departments. (pssst, btw, did ya spot that odd little grey balloon at the top of the second screen shot? Yeah, me too. It says "click allow if presented with a browser security message." Suspect, no doubt. Nonetheless, I trusted Chrome to protect me and plowed ahead). Razib continues his thoughtful discussion of the interplay of linguistic diversity/homogeneity and socio-economic disparity/prosperity.

Money quote:

If you have a casual knowledge of history or geography you know that languages are fault-lines around which intergroup conflict emerges. But more concretely I’ll dig into the literature or do a statistical analysis. I’ll have to correct for the fact that Africa and South Asia are among the most linguistically diverse regions in the world, and they kind of really suck on Human Development Indices. And I do have to add that the arrow of causality here is complex; not only do I believe linguistic homogeneity fosters integration and economies of scale, but I believe political and economic development foster linguistic homogeneity. So it might be what economists might term a “virtual circle.” (emphasis in original)

I have a long history of discussing language death on this blog and my position can be summed up by this Q&A I had with myself:

Q: Is language death a separate phenomenon from language change?
A: In terms of linguistic effect, I suspect not

Q: Are there any favorable outcomes of language death?
A: I suspect, yes (Razib proposes one)

Q: How do current rates of language death compare with historical rates?
A: Nearly impossible to tell

Q: What is the role of linguists wrt language death?
A: One might ask: what is the role of mechanics wrt global warming?
A new online version of the 2004 book A Valency Dictionary of English has recently gone live. I haven't had a chance to play with it, but it looks like it has some good data about verb patterns.If you're into that kinda thing, I mean. This is an awesome video of a Korean language professional teaching Korean speakers how to use swear words in English. It's so good, it's pickled.

(HT kottke) a pimple...

...and I remain happily ABD...

See The illustrated guide to a Ph.D. for full set of images and discussion. A high school math teacher helps us all understand critical thinking. Personal fav: Looks at statements that are generally false to see when they are true.

(via kottke) Hadas Shema, an Information Science PhD student, discusses some of the politics and problems with academic citations in The citation game. She got her facts from Bornmann, L., & Daniel, H. (2008). What do citation counts measure? A review of studies on citing behavior Journal of Documentation, 64 (1).

This one jumped out at me:

Journal-dependent factors: Getting published in a high-factor journal doesn't necessarily mean your paper is the best thing since sliced bread, but it means more people are likely to think so. Also, the first paper in a journal usually gets cited more often (I wonder if that's still relevant, given how wide-spread electronic access is these days) (emphasis added).

Lot's of crap has been published in major journals. And the corollary deserves to be mentioned: lots of good article ar published in minor journals and fail to get the respect or notice they deserve. The above image comes from the Stanford Dissertation Browser and is centered on Linguistics. This tool performs some kind of textual analysis of Stanford dissertations: every dissertation is taken as a weighted mixture of a unigram language model associated with every Stanford department. This lets us infer, that, say, dissertation X is 60% computer science, 20% physics, and so on... Essentially, the visualization shows word overlap between departments measured by letting the dissertations in one department borrow words from another department..

Thus, the image above suggests that Linguistics borrows more words from Computer Science, Education, and Psychology than it does from other disciplines. What was most interesting was using the Back button to creating a moving picture of dissertation language over the last 15 years. you'll see a lot of bouncing back and forth. Stats makes a couple jumps here and there.

HT Razib Khan Trey Jones* at Speculative Grammarian invites y'all to play his cute, and yet somewhat depressing, game: Choose Your Own Career in Linguistics.

As a service to our young and impressionable readers who are considering pursuing a career in linguistics, Speculative Grammarian is pleased to provide the following Gedankenexperiment to help you understand the possibilities and consequences of doing so. For our old and bitter readers who are too far along in their careers to have any real hope of changing the eventual outcome, we provide the following as a cruel reminder of what might have been.

Let the adventure begin...

*hehe, he used to work at Cycorp, hehe... ...then this is for you: The Penguin Classics Complete Library is a massive box set consisting of nearly every Penguin Classics book ever published and is available on Amazon for only (only!) $13,413.30.

A rundown:

• 1,082 titles
• laid end to end they would hit the 52-mile mark
• 700 pounds in weight
• 828 feet if you stacked them
• They arrived in 25 boxes

My only complaint would be that Penguin Classics tend to be crappy books physically.

HT Kottke.