Author's Opinion

The views in this column are those of the author and do not necessarily reflect the views of iTWire.

Have your say and comment below.

Monday, 08 February 2010 10:04

Learning with the computer using open source

An American developer studying in New Zealand is using the open source Python programming language to build an educational voice and vision dialogue system as part of his doctorate thesis.

John Graves, who ran a multimedia CD business during the dot-com boom and also put in time on Wall Street, is enrolled at the Auckland University of Technology (AUT). His doctorate aims to determine how quickly open source software projects can cycle or evolve.

The software system he is working on will be entered in the Chatterbox Challenge, a contest where one enters dialogue systems - a system where one types something into the system and the computer responds. He gave a talk about the system, called OpenAllureDS, at the education mini-conference during the recent Australian national Linux conference.
John Graves
Graves, a developer all his life, had three stints on Wall Street, the most recent ending in February last year when he was made redundant. "I left Wall Street in February and came down to New Zealand because once you're on the internet, New Zealand or New Jersey, it's the same difference. And the attraction of open source for me is, that the world is facing some pretty significant problems which really need to be addressed through knowledge," he says.

"The area of knowledge is where technology can make a very significant difference. Fourteen years ago, I started my own dot-com business. We developed multimedia CDs, and a product called 'PC Tours, your guide to the land of computers' was mass produced in about 1996. The goal at the time was to create a product based on HTML and screencast, screen recording technology. It had its own web browser on the disk, so all you had to do was slip it in and watch the content. This idea was a little ahead of its time and we didn't have the marketing resources to get ahead in that business.

"I then went to work where the money was, which was Wall Street. But I've always thought that something really needed to be done in this area. It was a really huge opportunity. With the current initiative, I'm kicking off a project to develop a new kind of educational interface. Instead of the computer interface - mouse and keyboard - we want to use the webcam and headset with microphone to give a voice and vision-enabled tutorial environment."


Said Graves: "As you're working on your command line in UNIX perhaps, the voice of the text speech generation can give you instructions about what to do. You might try it out. When you're ready to go to the next part of the lesson or receive some more instructions, you could just put your hand up, and that would essentially be the signal to the computer that you're ready to advance through the lesson.

"Or take the gestures that you make to choose selections from say, a multiple choice question that happens to be put on the screen. You would potentially, with the microphone, actually be speaking the responses to the question. This creates the opportunity for there to be an actual dialogue with the computer. I've got some sort of prototype software working, and a whole website of little Youtube videos demonstrating the progression of the development of this project."

At the Kiwi PyCon in November, Graves met Brian Thorne who opened up a code repository for the project. "At the moment it's the two of us," he says. "But I'm pretty confident that I'll have more collaborators by the end of this conference. And certainly as the word gets out, and people can get hold of the code and play with it, and see what's possible I'm hopeful that it's going to be one of these things that just take off."

Graves is hopeful that the idea of self-referential technology could be true of the educational technology he's working on, "but in an even wider sense, that when you have a system that can dialogue with a person in speech and gesture just as we're sitting here talking, that that's going to be a very compelling way to teach how to develop a system that dialogues with voice and gesture. And that the evolutionary improvement of a system like that should progress very quickly and propagate extremely quickly."

Growing up in Michigan, the car-capital of the US, it was but natural that Graves would develop an interest in technology. "As a teenager I became very interested in science and technology. I went up to Caltech. I dropped out from there after a while I shifted coasts and got a degree in Geography and Energy policy studies in New York City."

He worked for Morgan Stanley in Times Square on the trading floor. "At my previous job in San Diego, I'd been responsible for quantitative portfolios in a company that had about $US60 billion under management, and the quant group had about $20 billion. Very significant financial responsibilities, all of which are behind me now, because I'm seeing the opportunities in software and education to be so much greater and so much more significant."


His Caltech background was responsible for him landing the Wall Street job. "I had my resume in hand and I dropped the resume on all the tables at a job fair and one of them happened to be for a mutual fund company. The guy from that company looked through the stack of resumes and saw that I had Caltech on the resume. He was a physicist. And he thought, 'Oh, anyone from Caltech can do this job, we need to get this guy in here'."

Graves says the culture on Wall street is very much as it has been characterised and it's really something of a gambling culture. "There is a tremendous amount of money changing hands and consequently there's ways in which the brokerage system and the money managers can skim off their percent, percent and a half, two percent, almost without it troubling the clients. And so as a result, when the results being invested are in the $US20, $US40, $US60 billion dollar range, if you take one percent of that a year, that's enough to support quite a significant IT staff, portfolio management staff.

"For the years in which I worked in San Diego in management, my salary doubled every year for seven years, starting from a civil service background until I was making a healthy six figure salary. And of course that was duplicated when I went to New York."

He's not sure whether one can call Wall street people amoral. "They just take advantage of the situation. I've actually been on a trading floor when an airline-related accident would precipitate a sell-off in that particular airline stock and literally the traders were scrambling. They wanted to get that airline stock out of the portfolio as soon as possible in this one instance. Because they were so much on the ball, they actually managed to sell before the price decline took place."

As to his doctorate, he says he has just started. "But remarkably, using the Python development language, which I'm also just becoming familiar with, I've been able to pull together libraries of functionality that have addressed issues such as speech recognition, natural language processing, computer vision, and gestures recognition. It's just remarkable how the many pieces of this new interface are just lying out there, waiting to be assembled. And once they're all put together, it should facilitate the creation, through a very simple sort of question and answer scripting process, of dynamic tutorial systems."

This the third time Graves has tried his hand at educational software. "About six or seven years ago, I went to Sydney and enrolled there at the University of Technology Sydney, it was to study multimedia development in an educational software setting. They had a particular program there that would have given me the contacts and the capabilities to make another go at offering the product at that time.


"Things didn't work out. Actually, my marriage ended. I had to go back to the US. That's what led me to go back into money management and to working on Wall Street. This time around I'm determined to make it a success by incorporating many collaborators."

Graves said the LCA keynotes had made an impression on him. "Someone coming into this conference, it they attended nothing more than those keynote talks, would come away with an understanding of really the socially culturally significance of open source software development. Not only to this sort of dedicated group of people, but to the world in which we live.

"There are initiatives that are growing out of the work we are doing here, which are going to make the difference in how our world works in the coming years. Things that are happening here will save lives. They will improve the environment. They will help bring peace. I really believe that," he said, lips quivering and tears coming into his eyes.

Graves hopes that by next month, he will have a small team working on his project. And he hopes that by the time he gets to March 2011, there'll be a much bigger team. "We'll have a product that has been rolled out, translated into multiple languages, distributed around the world," he said.

Subscribe to ITWIRE UPDATE Newsletter here


The much awaited iTWire Shop is now open to our readers.

Visit the iTWire Shop, a leading destination for stylish accessories, gear & gadgets, lifestyle products and everyday portable office essentials, drones, zoom lenses for smartphones, software and online training.

PLUS Big Brands include: Apple, Lenovo, LG, Samsung, Sennheiser and many more.

Products available for any country.

We hope you enjoy and find value in the much anticipated iTWire Shop.



iTWire TV offers a unique value to the Tech Sector by providing a range of video interviews, news, views and reviews, and also provides the opportunity for vendors to promote your company and your marketing messages.

We work with you to develop the message and conduct the interview or product review in a safe and collaborative way. Unlike other Tech YouTube channels, we create a story around your message and post that on the homepage of ITWire, linking to your message.

In addition, your interview post message can be displayed in up to 7 different post displays on our the site to drive traffic and readers to your video content and downloads. This can be a significant Lead Generation opportunity for your business.

We also provide 3 videos in one recording/sitting if you require so that you have a series of videos to promote to your customers. Your sales team can add your emails to sales collateral and to the footer of their sales and marketing emails.

See the latest in Tech News, Views, Interviews, Reviews, Product Promos and Events. Plus funny videos from our readers and customers.


Sam Varghese

Sam Varghese has been writing for iTWire since 2006, a year after the site came into existence. For nearly a decade thereafter, he wrote mostly about free and open source software, based on his own use of this genre of software. Since May 2016, he has been writing across many areas of technology. He has been a journalist for nearly 40 years in India (Indian Express and Deccan Herald), the UAE (Khaleej Times) and Australia (Daily Commercial News (now defunct) and The Age). His personal blog is titled Irregular Expression.

Share News tips for the iTWire Journalists? Your tip will be anonymous




Guest Opinion

Guest Interviews

Guest Reviews

Guest Research

Guest Research & Case Studies

Channel News