|
This system allows the operation of UC-win/Road by moving the arms as operating
the steering in front of Kinect. The positions of both arms which are detected
by Kinect are converted into steering, acceleration, braking of vehicle.
Kinect™ sensor
What is Kinect Sensor?
Kinect Sensor (Hereinafter, Kinect) is used as a peripheral device of "Xbox360®"which
is a home-use game machine sold by Microsoft and is the device which allows
you to operate the system via physical behavior, gesture and voice without
a controller. A camera and sensor are mounted on it and players are automatically
recognized when they stand in front of the Kinect.
This technology is based on the depth camera technology developed by PrimeSense.
The sensor includes a VGA camera, 3D scanner, 4 microphones, inclined motor,
etc. The 3D scanner is composed of an infrared laser (whose frequency is
a little lower than that of a red signal) that gives off several dots in
a continuous pattern. Although the laser is not visible to the naked eye
(as the wave length is about 780nm), it can be detected by an infrared
camera(Figure 1a and 1b).
|
|
|
Figure 1a
A night-vision image of Kinect |
|
Figure 1b
Expanded view with points projected on the wall |
1. Output of the distance image
Kinect provides the distance image of 640 x 480 pixel (The distance which
is the closest from the sensor is stored per pixel.).
2. User detection and tracking
Kinect allows to detect and track the human within the visual field of the sensor.
3. Skeleton detection and tracking
Kinect allows to detect and track the joint position of the human.
AirDriving Interface
4.Air driving
Kinect allows to recognize the steering operation with both arms and the
acceleration / braking operation with the right foot and enables you to
operate the driving simulation by gesturing (steering, acceleration, braking
operation are applied in UC-win/Road).
|
Gesture driving simulation of UC-win/Road
via Kinect Sensor and 6K Multi-Cluster Digital Signage system
(Tokyo Game Show 2011) |
Vehicle control and imaging of driving operation via Kinect
Linkage with UC-win/Road enables you to experience the driving simulation
by holding the virtual steering wheel and moving the arms. It is possible
to keep the natural situation which is very close to the actual driving
operation.
- Neutral position
The operation of acceleration and braking is performed based on the distance
from Kinect to the arms. It can be operated by moving backward and forward
from the neutral position.
- Steering operation (Right-turning, Left-turning)
The steering can be operated same as actual steering control. It supports
for both of right-turning and left-turning.
- Acceleration operation
It can be accelerated by moving the arms from the neutral position to the
front. The further you move the arms, the larger the acceleration amount
becomes.
- Braking operation
The braking operation is performed by moving the arms from the neutral
position to the back. The further you move the arms, the larger the braking
amount becomes.
- Detection of steering angle and steering direction
The right and left direction of the steering wheel and its steering angle
are detected as an analog value from the positional relationship between
the user's right and left fist. How much the steering wheel is turned is
indicated by a row of bars that change color from green to red. The bigger
the steering angle, the more red the bar will be.
|
|
|
Travel in a straight line |
Right turn : As you steer to the right, the red bars will increase in the
right direction. |
Left turn : As you steer to the left, the red bars will increase in the
left direction. |
- Distinguishing accelerator from brake
Acceleration and braking can be distinguished by detecting the angles between
the right toe opening right and left.
|
|
Accelerator position |
Braking position |
- Detecting the amount of pressure on gas pedal and brake pedal
Accelerator
This is detected as an analog value from the amount of pressure the driver
exerts on the accelerator pedal. In case of accelerator, the stack of bars
will increase in the upward direction.
Brake
This is detected as an analog value from the amount of pressure the driver
exerts on the brake pedal. In case of brake, the stack of bars will increase
in the downward direction.
5. Virtual gear lever
Grasping behavior of objects and gear operation can be recognized (Moving
forward and backward can be switched in UC-win/Road).
6. Gesture Interface
Up to 12 kinds of operation can be distinguished by combining the operation
of right hand moving from right to left or up and down (Moving forward
and backward can be switched in UC-win/Road.).
|