Janison is not new; Wayne Houlden founded the business in Tamworth, NSW in 1998. Both he and his wife were teachers, and he'd already worked in IT for 15 years prior to teaching. Houlden's background in technology and education allowed him to see a skills gap in regional Australia where programs existed for networking and server administration but without access to certification. Thus Janison was born, with the Houldens working within the TAFE system to train and certify people.
The Janison system offered interactive tools, content management, and student discussion right from the very start of online learning. These were the early days of the web, with TAFE servers in storage rooms, and with Amazon and Google still emerging. Soon their courses were adopted across the state, then across the nation. The duo forged relationships with national education organisations and schools, but over time found keen interest from corporates and government too. Thus, the product continued evolving with expanded learning capabilities and support.
By 2010 the business had 30 employees and had relocated to Coffs Harbour but the online learning space was now a sizeable market with a growing range of enterprise players. Houlden recognised the need to scale if he was to compete and quickly hopped onto the public cloud as it appeared. This was a serious game-changer; up to that point, the Janison web servers were constrained by the expense and effort of maintaining the infrastructure required for peak periods all year long. The cloud appeared to be a perfect solution, offering scalable, elastic, on-demand computing power when needed. In fact, it was a pivotal moment.
Houlden turned the business to build online testing and measurement platforms whereby others performed the teaching but Janison's products would handle the testing - with its huge capacity requirements at times and none at others. One key partnership was the NSW Government's Essential Secondary Science Assessment (ESSA), taken by every year eight student that year. It was previously a paper-based science test; in 2011 Janison delivered the test electronically to 60,000 students on a single day, on top of Microsoft Azure.
The assessment journey continued, with Janison winning the NAPLAN online contract, the Singapore online exam system, the British Council's English assessment around the world, and OECD school measurement and improvement assessments.
"It's been a great ride," Houlden said. "In that process, we built our own tech and now owned some content as well," with Janison acquiring the ICAS business from UNSW Global. "We're proud we can continue to build and nurture that."
Other acquisitions followed, and by 2017 Janison was now an assessment business in its own right. It was not only running the test but also owned many of the tests and the data collected. A particular example is the ICAS core data around student performance in schools for writing, literacy, numeracy, science, and design and technology. ICAS is run every year, with data ranging from years one through 12.
What do you do with all that data? "It's an interesting place we find ourselves in," Houlden said. "We think tech can be a really positive agent for improving insight in education. Even now, as we sit in 2022, we still think about the business this way. How do we improve how people learn?"
"Measurement is our focus area. Accurately understanding how students are performing and learning is really important," Houlden said.
Janison's assessments are in use across the globe - Russia, Thailand, the United Kingdom, the United States, Brazil, Portugal, Spain, China, Kazakhstan, and more. Yet, this also means the data is no longer English-only. Janison's tests are in Russian, French, Chinese, and numerous other languages. The tests are carefully controlled so they are standardised across all countries. For multiple-choice questions, it is easy enough to map the correct response to each language, but for free-text questions, this is not so easy.
As an example, the test might ask the student to explain a passage in their own words, in three sentences. Around 30% of the OECD tests Janison administers are free-text responses. To solve this the organisation hired human scorers and, to ensure consistency, has assessed answers by different scorers to identify and resolve variations.
However, here's where it struck Houlden he could apply artificial intelligence to the challenge of accurately, consistently, reliably scoring human-readable text in a diverse range of languages. "The cost of scoring is quite high," he said. "With our work for the OECD, we have some countries with hundreds of markers engaged for four to five-week periods. It's a huge cost and a lot of expertise has to be specifically found in each country."
"Artificial intelligence is driven around a large amount of data," Houlden says. "In general, AI has to have something as an input to learn from. In the case of Janison, we have historical data."
Houlden and his team began investigating existing AI algorithms that could be taught how to score and found initial success where, if they had sufficient historic knowledge, they could score almost as accurately as a human.
The team began working on the problem, adapting as new algorithms emerged. This includes a Google algorithm named BERT pre-trained on textual data, and a Facebook cross-language algorithm named XLM-R using common crawl data with language contexts for 100 languages. Combining the great works of others in this field with their huge body of data, Janison has been able to train a model to score responses irrespective of the language it is in.
Significant work has been invested in dealing with bias, leveraging the work the company already performed in scoring responses multiple times for a proportion of responses, thus providing higher-quality data.
Houlden says their work is continually ongoing, models are updated on a regular basis, and new models are being developed. He credits Google, Microsoft, IBM, and others with building models and releasing many to the public domain.
"It's amazing how democratised AI has begun," Houlden said. "Software developers have access to these algorithms which we can use and explore and expand on, in a way that 10 or 20 years ago you'd never have imagined you'd get for free."