Take the word normal. In a maths lecture it can mean perpendicular to a surface. It can also mean the distribution, and in a third module it means a kind of subgroup, and which one you are looking at depends on which lecture theatre you happen to be sitting in. In an engineering module it is almost always the distribution. In a history essay it means usual, and using it any other way would read as a mistake. We are a maths student and an engineering student. We did not need a corpus study to notice this. We only had to compare lecture notes.
A single list of academic vocabulary has to carry one entry for a word like that. It cannot tell a student which sense their department uses, or which words it tends to sit beside. It cannot tell them whether their lecturers reach for it at all. When the question was put to us directly, whether there is such a thing as general academic vocabulary, the answer came out without hedging: it is context specific. Completely context specific. And subject specific.
What Hyland and Tse found, and who disputes it
That answer is also the finding of a paper we read afterwards rather than before. Hyland and Tse (2007), "Is There an 'Academic Vocabulary'?", in TESOL Quarterly 41(2), took Coxhead's Academic Word List and examined how its items actually behaved in academic writing across a range of disciplines. They did not behave as one set. The frequencies came out different from discipline to discipline. So did the sense that got used, and so did the company each word kept. Fields even differed on which member of a word family they preferred. Their conclusion was that a student is better served building a lexical repertoire out of the texts of their own field than working through a list assembled to serve everybody at once.
We should say plainly that this is contested, because anyone who works in the area already knows it is. Gardner and Davies (2014) built a different academic list from a different corpus and drew a more general conclusion from it, and Durrant (2016) went at the relevance of that list to student writing in English for Specific Purposes. The disagreement turns on corpus design and on how much shared vocabulary counts as enough to justify a common list. We are not in a position to settle it, and we would be suspicious of a company that claimed to be.
What we can say is that we are leaning on the side of the argument that happens to suit our product, and that we would probably have leaned on it either way. That is a reason for a reader to discount our enthusiasm rather than a reason for us to feel vindicated.
Our vocabulary comes from what the student uploads
The vocabulary in our product does not come from a list. A student uploads their own course material, a set of lecture slides or a reading, and the vocabulary is drawn out of that. The original reason for this was not theoretical. We assumed it would be the most useful thing we could do, because a student is going to need those particular words to do well on that particular course. That is an argument about assessment rather than about correctness, and it happens to sidestep most of the question of whose English is being taught, because the English being taught is the English of the student's own department, written by the people who will be marking them.
On this one feature, the English a student meets is their department's rather than ours.
Provision often runs the other way. A bought-in textbook teaches a version of academic English that belongs to no department in particular, and generic provision of that kind is part of why English for Academic Purposes gets treated as a service bolted to a degree rather than as a subject with something of its own to teach. Ken Hyland (2018), in Language Teaching, wrote a defence of the field against precisely that treatment. Paul Ashwin's line is the compact version of the problem: "skills without knowledge is no skill at all". Extraction from a student's own material is a direct answer to it, and it is the part of what we have built that we would defend hardest.
One of the two students who sent us written feedback named it before we asked her to. She had used the product for about two days, and her account of what was different about it was that it "allows students to learn vocabulary related to their major and generate related vocabulary using their own courseware". Two students, one of them over two days, is not evidence that anything works, and we will not pretend otherwise. What it is, is a user with no interest in our theory identifying the same feature we would have pointed at. The same student wrote, of Chinese students generally, that "we only remember and spell but don't know how to use it", which is the objection to word lists arriving from someone who has never read Hyland and Tse.
The part that costs us
The disciplinary claim covers vocabulary and nothing else. The uploaded course material feeds the extraction and stops there. It does not reach the conversation practice, and it does not reach the writing feedback. Those two features respond to what a student produces and to how much help they turn out to need, which is responsiveness to a student's level rather than to their subject. Lea and Street (1998), in Studies in Higher Education 23(2), describe academic literacy as plural and specific to a field rather than as one transferable skill, and on that description neither feature is disciplinary at all. They are generic provision that happens to be responsive.
So the accurate description of the product is that it has a disciplinary component rather than that it is a disciplinary product, and most of what a student spends their time on is the generic part. This is buildable. We have not built it. We are not going to describe it as a roadmap item and move on, because the shape is wrong now, and anyone reading our marketing would come away with a stronger impression than the one the code supports.
The problems we know about in our own decks
There is a second problem with the vocabulary feature itself, and it points the other way from everything above. Nakata and Suzuki (2018), in Studies in Second Language Acquisition, report that words learned in semantically related sets are learned less well than words learned in unrelated sets. A deck built from a single module is related by construction. That is the entire point of it, and it may also be a cost we are imposing without noticing. The literature does separate two kinds of relatedness: words related in the narrow sense, the way synonyms or the members of one category are related, and words that merely cluster around a topic. Tinkham (1997) reported interference for the first kind and something closer to help for the second, which would put a course-derived deck on the safer side.
We are not going to hide behind that. We have never inspected our own decks in those terms, we do not order or space items with the finding in mind, and we do not know which kind of relatedness a lecture on thermodynamics or on measure theory actually produces. It is a live risk in the feature we are proudest of and we have done nothing about it.
A few smaller admissions belong together. Extraction returns single words, and a good deal of disciplinary meaning sits in units longer than a word, so the thing we return is often not the thing that carries the meaning. Extraction also leans on how often a word appears, when how widely it is spread across a text matters at least as much, and a term that dominates one slide is not the same as a term the whole module runs on. And a word paired with a definition leaves a student with word association to an extent, whatever corpus the word was drawn from. Sourcing vocabulary from the right place does not repair the unit. It only means the wrong unit is at least made of the right material.
How big should the corpus be?
What is still open is the size of the corpus, and we do not have a principled answer. Hyland and Tse's unit is the discipline. Ours is whatever a student happened to upload, which in practice is a module, and sometimes a single slide deck written by one lecturer with their own habits. A module is not a discipline. A lecturer is not a field. It is possible that a corpus that small teaches a student the vocabulary of their course, and possible that it teaches them the vocabulary of one person who is going to mark them, and those are not the same achievement even though they would look identical in a term's results. Nobody has used the product long enough for us to tell the difference. Nothing in it currently notices there is a difference to tell.
