The talking kitchen that teaches you French

Supplementary content information

An innovative kitchen that gives step-by-step cooking instructions in French could spark a revolution in language learning in the UK.

You must select the video player for these keys to function.

Keyboard shortcut Function
Spacebar Play/Pause when the seek bar is selected. Activate a button if a button has focus.
Play/Pause Media Key on keyboards Play / Pause.
K Pause/Play in player.
Stop Media Key on keyboards Stop.
Next Track Media Key on keyboards Moves to the next track in a playlist.
Left/Right arrow on the seek bar Seek backward/forward 5 seconds.
J Seek backward 10 seconds in player.
L Seek forward 10 seconds in player.
Home/End on the seek bar Seek to the beginning/last seconds of the video.
Up/Down arrow on the seek bar Increase/Decrease volume 5%.
Numbers 1 to 9 on the seek bar (not on the numeric pad) Seek to the 10% to 90% of the video.
 
Number 0 on the seek bar  (not on the numeric pad) Seek to the beginning of the video.
 
Number 1 or Shift+1 Move between H1 headers.
/ Go to search box.
F Activate full screen. If full screen mode is enabled, activate F again or press escape to exit full screen mode. 
C Activate closed captions and subtitles if available. To hide captions and subtitles, activate C again. 
Shift+N Move to the next video (If you are using a playlist, will go to the next video of the playlist. If not using a playlist, it will move to the next YouTube suggested video).
Shift+P Move to the previous video. Note that this shortcut only works when you are using a playlist. 

[Computer talking French]

[Narrator]

If conjugating verbs isn't your idea of an interesting way to learn a language, the latest research at Newcastle University could be just the thing for you. Language experts and computer scientists have developed an innovative kitchen that gives lessons in Cooking and French at the same time. It tracks your actions with motion sensor technology, similar to a Nintendo Wii, and it speaks to you like a car sat nav as you prepare your dish. Professor Paul Seedhouse is involved in the linguistic side of the project.

Paul Seedhouse - Newcastle University [PS]

This research has been based on previous research done by my colleague Patrick Olivier in Computing Science in Newcastle. He developed a digital kitchen for people with dementia. So for example, if they left the oven on for too long, it would tell them to turn it off and when I saw that, I thought that the technology could be adapted for use with language learning and teaching.

[Computer talking French]

[PS]

When you enter the kitchen, it looks pretty much like an ordinary kitchen. Some of the differences you'll notice if you look at the kitchen equipment, knife and spoon, the handles are larger than normal and that's because they have sensors embedded. This technology you may be familiar with already, it's similar to Nintendo Wii technology, so there are sensors and they detect movement in all directions. The kitchen will then speak to you in French and that's similar to sat nav which you are probably familiar with already in your car.

[Computer talking French/English]

[PS]

When you're actually doing the cooking, you have a number of alternatives open to you. You can request a repetition, ask for a translation, pause the programme or you can go backwards and forwards. After you have finished cooking, we give you some tests and these show us whether you have learnt the vocabulary and the grammar items which we have targeted. But a really important point about this learning environment is that we put people in pairs. Typically, one person is better at French, one person is better at cooking and that means that they transfer the skills and they help each other to learn.

[Computer talking French]

[PS]

The recipe we've got is called Pear Clafoutis and it tells you first of all to peel the pears. So it tells you to take four pears and the system then detects that you've taken the package of pears and you've moved it onto the work surface. It then tells you to take the peeler and the sensor in the peeler will then detect that you've taken it over, it's been moved, and then the system will detect whether you have made the correct peeling motion with your peeler. When you've done that it will tell you to cut the pears into four. It will then check that you have taken the knife - has the knife moved over to the surface and then have you made the correct chopping motion with the knife. If you don't make a chopping motion with the knife, let's say you move it from side to side, it won't detect that as correct and so the system will loop back and give you the instructions again.

[Narrator]

Dan Jackson and Jurgen Wagner are involved in the technical aspects of the project. They explained more about the small sensing devices that are embedded in the kitchen utensils.

Dan Jackson - Newcastle University [DJ]

Fundamentally, we are interested in understanding how the objects are used and how they are being manipulated. We've developed this hardware that actually detects the motion, uses something called an accelerometer which is a device that measures acceleration.

Jurgen Wagner - Newcastle University [JW]

The accelerometer inside tracks the smallest movements you do with your hands, the smallest movement you do with the utensils. These are called motion primitives and this information is sent over to a computer which acts as a base station which translates these simple activities, these motion primitives, into something more meaningful such as was it a chopping activity, a stirring activity or whatever activity we can measure.

[DJ]

So for instance, if you pick up a utensil, the device will wake up from a sleep state and it will start broadcasting its motion to the computer. Then on the computer, we can analyse that information and work out what that object is being used for. Chopping with a knife would give us a different signal to someone scraping or slicing with the same knife.

[JW]

Depends what you want to measure. If you want to measure that the person is following the recipe then you can track the user through the recipe seeing when/which activity occurs, was the activity at this current moment the activity we expected, or was it a totally different activity and you can use this then to prompt or guide the user through the recipe and check if it's been done correctly.

[Computer and students talking in French/English]

[Narrator]

As well as making learning fun, Paul Seedhouse says the kitchen builds on the proven technique of task based language learning where students are prompted by instructions in the language they are studying to carry out activities. There are now further plans to take the concept out to a wider audience.

[PS]

Well, on the point of view of language learning and teaching, it's unique because it's taking the excellent ideas of task based learning and teaching out of the classroom and, if you like, into the kitchen so you are combining it with a real world task. It's also unique in that you are learning two skills at the same time: in other words French skills and cooking skills. We already have the full scale digital kitchen in Newcastle University, but we have now developed a series of portable kitchens which we take out on road shows. It's basically a tablet PC, six implements and a pair of speakers, so it's very portable and we are wanting to put those into schools in the North East. We've also got a big grant from the European Union and we are going to be placing portable kitchens in four different countries in Europe and trying them out there.

[Narrator]

The research is supported by the Digital Economy programme run by the Engineering and Physical Sciences Research Council. In the future this way of learning could be used to teach people any number of skills ranging from nursing to driving. The new type of kitchen could be available for schools, universities and even the domestic market by the end of 2012.