Academy Users Report / Vol.37

Tsukada Lab., Department of
Business Administration,

Setsunan University

Metaverse projects developed through problem-solving learning by mixed-grade seminar teams
Exploring the use of F8VPS in infrastructure maintenance and management in joint research between industry and academia


Tsukada Lab., Department of Business Administration, Setsunan University
URL https://www.setsunan.ac.jp/
Location: Neyagawa City, Osaka
Research Contents: Creation of business plans and solutions for solving social and corporate issues by utilizing information technology such as AI





Assoc. Prof. Yoshinori Tsukada,
Tsukada Lab., Department of Business Administration, Setsunan University

"Designing business plans to solve social and corporate issues by skillfully using information technologies such as AI and the metaverse. Thinking about such things is the main theme of the seminar students in the Tsukada Laboratory." Since Assoc. Prof. Yoshinori Tsukada joined the Faculty of Business Administration in 2019, he has continued with this kind of initiative, which is unique for a seminar in the university's Faculty of Humanities.

When he was an undergraduate student at Kansai University in 2009, he joined Kansai Information Institute Co.Ltd. (KII), a student startup company established by the Shigenori Tanaka Lab. of the Faculty of Informatics, of which he was a member. He received a PhD at the University's Graduate School, specializing in intelligent informatics (AI: Artificial Intelligence), perceptual information processing (computer vision, image processing), and social infrastructure informatics. He took up his current position after working as a specially appointed professor at the Organization for Research & Development of Innovative Science & Technology (ORDIST) at Kansai University in 2015 and as a lecturer at the Faculty of Software and Information Science at Iwate Prefectural University in 2016.

"The subject of my doctoral dissertation was categorized as civil engineering informatics. With the handling of 3D data as a continuous theme, I specifically studied how to utilize 3D data of bridges, roads, etc. generated using images from laser scanners and cameras for maintenance", says Prof. Tsukada. At the KII mentioned above, he was involved in the development of applications and components using web technology, various measurement systems using advanced image processing and photogrammetry technology, and the application of CAD/GIS technology, and this wealth of business experience in the private sector is used in his seminar projects.



Design business ideas through the Metaverse

Students at Setsunan University are assigned to seminars as early as their sophomore year, and participate in practical projects while deepening their respective fields of study. The Tsukada Laboratory Seminar has 45 students, and each mixed-grade project team, consisting of sophomores to seniors, works on PBL (Project Based Learning) such as "Jobhunting Support by AI", "Metaverse", "Smart Kitchen", "Eye Tracking", "People Flow Analysis", and "Road Inspection". The seminar is characterised by practical and proactive learning. For example, each team takes charge of a local government or private company and develops activities to meet their needs, and once the year's schedule is decided, the students take the lead in progressing the projects.

Mr. Tsukada says that the use of 3D data is essential in giving shape to various ideas. In the "Metaverse" project, he introduced "F8VPS", a WebVR platform system from FORUM8, to explore its actual use in attracting customers to events and solving corporate problems.

"Since my students are studying in a kind of humanities faculty, we are working on projects together with the students of Prof. Shigenori Tanaka's seminar, my former teacher at the Faculty of Informatics, Kansai University, with help from them on scientific knowledge and technology such as advanced system development in the process of realizing ideas."

Prof. Tsukada first used the UC-win/Road 3D real-time simulation software when he was a member of the Tanaka Laboratory, and already felt its usefulness in the visualizing 3D information. In recent years, when the metaverse began to attract social attention, Prof. Tanaka introduced him to the "F8VPS" WebVR platform system, which can build the metaverse, and the integrated CG software Shade3D, which can be used for spatial modeling of the metaverse, and he decided to introduce these products with a view to using them in the projects in his seminar.


Seminar members (seniors: 12, juniors: 18, sophomores: 15). They join the seminar at an early stage to be involved in practical projects in their areas of specialization.


Deploying metaverse projects for various purposes such as personnel/recruitment, digital archives, job hunting support, etc.

The Metaverse project teams at the Tsukada Laboratory Seminar are working on several projects in parallel. One of them, the "Metaverse Human Resources" project team, is using the F8VPS Metaverse in cooperation with NIPPON INSIEK CO.,LTD., a construction consulting company, to solve various limitations in corporate recruitment activities. Not only does it have a "web conferencing function", which allows HR staff to give a presentation at the company information session in the Metaverse space and participants to listen to the explanation and receive materials via screen sharing, but it is also effective in creating communication opportunities for students to talk directly with management who are usually too busy to talk with, or with young employees and job offers by using avatars and the "standing conversation function".

Since the "Metaverse HR" is free from time and space constraints, there would be many advantages. For example, it is possible to recruit various human resources from all over the country, and to exchange human resources with foreign countries through AI mediation even if the employees do not speak English.

In addition, as a unique benefit of the Metaverse, we are also creating content that reproduces spaces that even interns cannot actually visit, such as highway bridge inspections, so that they can participate remotely and experience work simulation. The project team is considering a system that allows students to experience the real business, such as creating a 3D model of a bridge inspection vehicle that, when clicked on, plays a related video explaining the project, and making the underside of the bridge assuming a junction, etc. with the rust and cracks that are in the inspection items being reproduced, which the actual inspection being displayed when clicking on the damaged part.

Students who actually use the F8VPS in their projects say that the functions of conventional web conferencing tools, which are available in 3D space as they are, and various communication functions such as chat are useful.

In another project, one of the teams is trying to use AI and Metaverse to support the employment of people with disabilities, which has actually developed into collaboration with a social welfare organization in the Kansai region. Even if people have physical limitations in the real world or are not good at speaking in front of others, they can use avatars in the metaverse space to move around freely and get their work done. "Currently, students are creating spatial content for the metaverse, but if people with disabilities can create such content under contract from companies, it will be helpful from an employment support perspective. Some of them have built up self-confidence while working freely in the metaverse and have got jobs in the real world."

Other metaverse business plans they have planned and presented include "Metaverse Real Estate", a collaboration project with a real estate agent that uses point cloud surveying to create a highly accurate spatial model of a property to simulate furniture placement and loading/unloading, "Metaverse Museum", which attempts to monetize digital archives (heritage, records and folklore), and "Virtual Campus", a virtual open campus created from point cloud data of Setsunan University, which allows high school students to attend mock lectures and experience campus life through a walkthrough from their home or school.

You can also hold briefings using the web conferencing function and communicate with each other using the stand talk function.
They created a platform in the metaverse to simulate and explain the real work that cannot be experienced even during the internships. Realistic experience is possible by showing videos explaining how real inspections are carried out (Metaverse Personnel project).


Highway maintenance management by linking point cloud data and metaverse

The metaverse is used in the seminar to solve business problems and realize business ideas. Mr. Tsukada himself is participating as an extramural research cooperation member in the Infrastructure Management Research Group, which was established in 2022 primarily at Kansai University, and is working on the management of data on highways and other public structures based on the metaverse space and the verification of its usefulness. In addition to Kansai University, FORUM8, SUBARU enterprise Co., Ltd. and NIPPON INSIEK CO.,LTD. are participating in this study.

NIPPON INSIEK conducts the high-precision 3D laser survey of the highways, and FORUM8 creates the metaverse space using the point cloud data obtained. Not only do they create structure models such as lighting and power poles that are associated with roads and need to be inspected to run various simulations, but they also aim to visualize attribute information and create a maintenance platform. Since SUBARU is a company which undertakes actual inspections and maintenance, it is participating in the project as a supervisor to evaluate benefits of the spatial data management. At the moment we meet regularly to exchange views, discuss topics and come up with concrete ideas.

The working group in this project, in which Mr. Tsukada participates, collects point cloud data and 3D models of public structures, as well as design drawings, inspection forms, site photos, etc. generated in each process of survey, design, construction, and maintenance management. Using this data, they propose methods for managing and visualizing digital data in 3D metaverse space based on position and reference data.

"Potholes and cracks in the road surface are found during daily patrols, and the inspection results and information are summarized in 2D data such as Excel. Normally, when you acquire images and data, you probably manage them in folders on your desktop in two dimensions. This research group is trying to manage and refer to this data efficiently by associating different data in 3D space."

Regarding this, they will discuss what kind of functions are necessary for registering information by the end of this year, and once the specification policy is decided, FORUM8 will develop F8VPS, and it is expected that the relevant parties will start using it.

In addition, with regard to the inspection of pavements and accessories in the metaverse space, they are verifying inspection methods that can be performed in the metaverse while looking at the information that is constantly coming in from the field. Specifically, the system uses AI to automatically assess damage conditions, such as where and how the road surface is cracked, from video captured by drive recorders on road patrol cars, and the results are automatically reflected in the metaverse space based on the location information.

An ongoing project by the Infrastructure Management Research Group, led by Kansai University, involving Setsunan University and FORUM8, to automatically inspect and manage data on public structures in a metaverse space. They are creating 3D models from point cloud data of highways and considering a maintenance management method using the metaverse. In the future, the research group's activities will be presented at symposia organized by the "Journal of Digital Life", an academic article site in the digital domain (https://journal-digitallife.com/).


Expectations for the WebVR platform that turns free ideas into businesses

"Today's young people, who have absolutely no resistance to 3D, are not only losing the awareness that they have to be face-to-face, but rather are thinking about what they can do only online. In order to establish their free idea as a business, it is attractive that you can build your own metaverse platform and design it in detail according to your purpose with a high degree of freedom", said Mr. Tsukada, talking about the usefulness of the F8VPS as a method for practically solving social and business problems. The Tsukada Lab. seminar has a policy that the members proactively submit business plans created by students to external idea competitions, academic societies, or research groups at least once a year, both as a team and individually. They have accumulated a large number of awards every year, and it is expected that the F8VPS will be used as a tool to further support these activities.

"A number of projects related to the metaverse are currently underway. The students are taking action on everything they can think of at the moment, and in the future, I would like to customize and expand the platform to complete each project."

Part of this project is supported by the JST Strategic Basic Research Programs JPMJRX21I2.


Their active student-led project activities have achieved various successes and have been featured
in the university's public relations magazine.
(Up&Coming '23 Summer issue)



Previous
  
Index
  

LOADING