Stage recherche à Pau H/F CESI

Title: Hybrid Collaborative XR Environment Operated with Digital Twin in Edge-Cloud.

Ing./MSc. Intership Proposal in Computer Science

Scientific Fields: Human-Computer Interaction, Edge/Cloud Computing, Extended Reality and Digital Twin Technologies

Keywords: Augmented Reality, Cloud, Docker, Data compression, Edge, Extended Reality, Digital Twins, IIoT, Machine Learning, Unity, OPCUA, Virtual Reality

Supervisors:

• Hugues M. KAMDJOU, Researcher at CESI-LINEACT

• Havard Vincent, Associate Professor at CESI-LINEACT

Research Work

Internship Description

EXtended Reality (XR) alongside the Digital Twin (DT) in Industrial Internet of Things (IIoT) emerges as a promising next-generation technology. This project centers around advancing collaborative XR experiences by seamlessly integrating Virtual Reality (VR) and/or Augmented Reality (AR), fostering a cohesive experience for both location-based and remote users. In the initial phase of the project, we have already developed an environment integrating VR with DT. The current aim is to extend this immersive experience by integrating AR within the framework of an Edge-Cloud architecture. This includes maintaining consistency in the hybrid collaborative environment while incorporating the unique features of AR. Building upon the developed environment, the internship will also explore how data compression can be further utilized to enhance the realism and dynamism of both VR and AR elements within the XR space coupled with DT.

Previous Work in the Laboratory: JENII project, is a remote learning initiative for the future industry, built upon immersive and collaborative environments centered around digital twins of real industrial systems.

Work Program/Objectives: The main aim of this Master's thesis is to enhance Quality of Service (QoS) of hybrid collaborative XR experiences through synchronized DT in Edge-Cloud. The objectives/program include: 1. Implementation of an Edge-Cloud architecture to optimize performance and reduce latency of the existing XR environment. 2. Integration of automated monitoring using the camera-equipped robotic arm to capture the state object conditions and transmit data to the DT; integration of multi-user and augmented reality into the existing XR environment. 3. Test a multi-user collaboration scenario on a specific use case and evaluate the QoS. 4. Propose potential solutions to integrate data compression into the framework for reducing latency. 5. Propose potential solutions to establish common ground among co-located and remote users with heterogeneous devices.

Expected Scientific/Technical Output: This internship project bridges research and industry, driving progress in immersive technologies and industrial efficiency. The purpose is to investigate performance optimisation techniques for hybrid collaborative XR environments. The expected outcomes include a comprehensive documentation report delineating the research, experiments, and evaluations conducted during the proof-of-concept development phase. These contributions serve to propel advancements in XR and DT technologies, enriching the academic community's knowledge base and catalyzing innovation. Industrial partners stand to reap operational advantages from enhanced performance in real-world XR applications.

Context

Laboratory Presentation:

CESI LINEACT ("Laboratoire d'Innovation Num´erique pour les Entreprises et les Apprentissages au service de la Comp´etitivit´e des Territoires") Tools and Digital Engineering (TDN) team conducts research activities in human-machine interactions, XR, applied robotics, digital twins, fusion of information and decision in cyber physical production system. Recent works are interested in human activity/action recognition based on deep learning approaches for human-robot collaboration applications. The approaches explored also the generation of data for human action recognition from the digital twin associated with virtual or augmented environments allowing to simulate industrial scenario involving human robot interaction. CESI LINEACT also have various research facilities like flexible manufacturing assembly with cobotic or dual-arm TIAGO++ mobile robot associated to their digital twins which could be used in these experiments.

Positioning in Laboratory Research thematics: The project aims to harness the synergies between Modeling, desing and architecture of CPS, and Collaborative processes and digital tools.

Thesis/Internship Organization

• Workplace: CESI LINEACT, Campus PAU, 8 rue des Fr`eres d'Orbigny 64000 Pau, France.

• Start Date: As soon as possible until it is filled. 2

• Duration: 5/6 months

References

[1] Kamdjou Hugues M., Baudry David, Havard Vincent, and Ouchani Samir. Resource-constrained extended reality operated with digital twin in industrial internet of things. IEEE Open Journal of the Communications Society, 5:928-950, 2024.

[2] Vincent Havard, Alexandre Courallet, David Baudry, and Hugues Delalin. Virtual reality and opcua-based architecture for pedagogical scenarios in manufacturing and computer sciences curriculum. in Proceedings of the 13th Conference on Learning Factories (CLF 2023), 2023. 3

[3] Shakarami A., Ghobaei-Arani M., and et al. Masdari M. A survey on the computation offloading approaches in mobile edge/cloud computing environment: A stochastic-based perspective. J Grid Computing, 18:639-671, 2020.

[4] Yi Ding, Weiwei Fang, and et al. Mengran Liu. Jmdc: A joint model and data compression system for deep neural networks collaborative computing in edge-cloud networks. Journal of Parallel and Distributed Computing, 173:83-93, 2023.

[5] Yu S., Sun S., Yan W., Liu G., and X. Li. A method based on curvature and hierarchical strategy for dynamic point cloud compression in augmented and virtual reality system. Sensors, 22:1262, 2022.

[6] Joseph Azar, Abdallah Makhoul, Mahmoud Barhamgi, and Raphael Couturier. An energy efficient iot data compression approach for edge machine learning. Future Generation Computer Systems, 96:168-175, 2019.

[7] Wang Y., Chakravarthula P., Sun Q., and Chen B. Joint neural phase retrieval and compression for energy- and computation-efficient holography on the edge. ACM Trans. Graph, 1, 2022.

Recruitment / Candidate Profile

Modalities:

File review and interview. All qualified individuals are encouraged to apply by sending to (hmkamdjou [at] cesi.fr; vhavard [at] cesi.fr, with the email subject: "[Application] Title mentioned on page 1". a cover letter, a resume, transcripts of M1 and the current year of M2 (or equivalent level), BSc/MSc/Ing. certificates and at least two recommendation letters. Applications will be processed as they arrive, early application is highly encouraged.

Application should include:

• Detailed Curriculum Vitae of the candidate. In case of a break in academic studies, please provide an explanation;

• A motivation letter explaining your motivations for pursuing a doctoral thesis;

• Transcripts of MASTER I and/or II and/or corresponding grade reports;

• BSc/MSc/Ing. certificates;

• Two recommendation letters.

Please submit all documents in a zip file titled FIRSTNAME LASTNAME.zip.

Skills:

The candidate should possess a Master student or equivalent in Computer Science or Applied Mathematics. She/He should have some knowledge and experience in a number of the following points:

• Scientific and Technical Skills:

- Solid programming and software development tools skills (Docker, Node JS, C# and Unity 3D),

- Strong interest in extended reality (XR) technologies, digital twins, edge computing, and cloud architecture,

- Familiarity with networking concepts and protocols, particularly in the context of edge computing and cloud architecture,

- Effective communication skills in English/French and the ability to collaborate within a multidisciplinary team environment.

• Interpersonal Skills:

- Being autonomous, having initiative and curiosity,

- Ability to work in a team and have good interpersonal skills,

- Being rigorous.

CESI est une école d'ingénieurs qui fait de la promotion sociale par l'excellence un modèle de réussite. Rejoignez un environnement stimulant où l'esprit d'équipe, la diversité des projets et l'autonomie ne font qu'un. Découvrez une école qui a su développer un modèle unique et se donne les moyens au quotidien de relever les grands défis de l'époque. Nos 25 campus, 28 000 étudiants, 8000 entreprises partenaires et 106 000 alumni témoignent de l'impact de CESI au niveau national.

CESI accompagne ses étudiants en utilisant des méthodes innovantes de pédagogie active. L'établissement forme avec rigueur les futurs ingénieurs, techniciens et managers, dans les secteurs suivants : l'Industrie & l'Innovation, le BTP, l'Informatique et le Numérique et le Développement Durable. Parallèlement, CESI concrétise son engagement dans la Recherche à travers des activités menées au sein de son Laboratoire d'Innovation Numérique, CESI LINEACT.

Les partenariats établis avec 130 universités à travers le globe, attestent de l'engagement international de CESI. Ces liens privilégiés offrent aux élèves ingénieurs une mobilité sortante et entrante à l'échelle internationale, façonnée notamment par des stages obligatoires faisant partie intégrante de leur cursus.