PROJECT OUTLINE
Designed with great attention to details and user-centric functionality, the jig is designed to aid users of all technical expertise in producing the highest quality scan possible.
Its circular design ensures continuous centering of the plant within the frame, guaranteeing optimal scan results. Built to be secure and intuitive, users can confidently glide their mobile phones along the jig rails, knowing their scans will be captured with precision and clarity.
This section of our project comprises the 3D scanning technology we utilised in our project.
Our team researched on various methods of creating 3D renders of object.
We started off experimenting with widely available 3D scanning apps that had been built upon photogrammetry technology as we wanted our scanning method to be accessible. We then looked into learning-based scene rendering apps and models, and ultimately landed on a technology called Gaussian Splatting, which uses a neural-network based renderer to train and produce high quality 3D scenes.
The final section of our project which ties together our entire pipeline would be the archive.
The archive is a web application that consists of a frontend user interface connected to a backend database or archive.
After taking a video scan of the plant, users will upload their video files to our web application. These video files will be trained with the Gaussian Splatting script and the output splat files will be stored in our database. When a user wishes to view the plants in our archive, the file and information will be retrieved from our database and be displayed on the website.
THE JIG DESIGN
Designed to streamline user’s 3D Scanning process, this tool offers effortless usability and exceptional scan quality.
Crafted with a circular design, the jig ensures that the subject remains perfectly centered in the frame, allowing for consistent and accurate scans every time.
From seasoned professionals to novice enthusiasts, we designed the jig to be accessible and intuitive to all. In line with our commitment to environmental responsibility, the fabrication of our jigs also prioritize sustainability so that our jigs can contribute to conservation efforts without compromising the environment.
The phone mount glides effortlessly along the circular ring-like tracks, with the help of the smooth-wheel operation.
This purposeful engineering allows for fluid movement, eliminating any potential for jerky videos, thereby ensuring optimal clarity in the captured footage free of disruptions or distortions. This is essential in preserving the integrity of the captured videos, laying the foundation for precision in the subsequent image stitching Gaussian Splatting process.
The assembly of our jig was also designed to be a user-friendly experience, crafted for accessibility and ease of use.
Each component is carefully designed to seamlessly fit together, ensuring a hassle-free set up process. With intuitive design features and straightforward instructions, users can quickly and effortlessly assembly their very own jig without the need for specialized tools or expertise. Our emphasis on simplicity and efficiency means that users can spend less time on setup and more time capturing scan results with confidence.
At the crux of our design lies a deep understanding of the importance of precise orientation in the 3D scanning process.
The deliberate alignment of the jig’s tracks towards the subject – centered around the plant – captures the importance of optimizing the scanning process for unparalleled accuracy and efficiency.
In Gaussian Splatting, maintaining the subject in the center of the frame is crucial as it relies on projecting 3D points onto a 3D image plane using camera parameters. If the subject is not consistently centered, it can result in distorted or inaccurate projections, leading to errors in the 3D reconstruction. Consistent centering also ensures uniform coverage of the subject from all view points, essential for capturing all details and angles accuracy and eventually for achieving a high quality final 3D scene.
3D RECONSTUCTION
Gaussian Splatting
Our 3D scanning technology utilised a learning-based scanning technology called Gaussian Splatting, which uses a machine learning model to stitch and fill in gaps between pictures to render a high quality 3D scene.
Gaussian Splatting serves as a pivotal component within our pipeline for 3D scene reconstruction, seamlessly integrating into our system to convert user-submitted videos into detailed and immersive 3D representations.
When a user submits a video to our website, it undergoes 3 key processes:
- 3D Reconstruction
- Training
- 3D Rendering
Upon receiving a video file from a user, our script initiates the pipeline, converting the video into photos. The photos are then reconstructed and mapped to a point in the 3D space. Finally, the model can train these images, filling in the gaps of information between two photos.
This process produces a point cloud. The process is not over yet as the point cloud needs to be converted into a splat file. This splat file is sent to the database to be retrieved by the website and rendered on the website.
Scan Results
Utilizing Gaussian splatting for 3D plant scanning yields exceptional results, characterized by precise fidelity and detail.
This technique excels in capturing intricate features such as leaf structures, textures, and contours with remarkable precision and accuracy. By minimizing noise and maintaining consistency in data capture, Gaussian splatting ensures that each aspect of the plant is clearly represented in the resulting 3D model.
The presence of our final jig further enhances these outcomes, providing users with a guided scanning experience that minimizes shakiness and ensures optimal video clarity. As a result, images derived from the video exhibit improved alignment and coherence, facilitating seamless stitching and reducing noise levels in the final 3D scans.
These are out results.
ARCHIVE
User Interface
From the initial stages of constructing the information architecture and wireframes of the website, to prototyping and final development, the Bios Archive website was designed with our potential users in mind.
Be it the browsing users who just wish to view a plant or users who are plant enthusiasts or botanists who are interested in making a contribution to our archive, we aimed to provide them with the best user experience possible.
With the backbone of the website’s structure and frontend design being built with Next.js and Tailwind CSS, Three.js was seamlessly integrated with the React-Three-Fiber library to bring our 3D plant scenes to life, ensuring smooth rendering and immersive and interactive user experiences.
Website Backend
The backend of our archival website needs to play the role of storing user information, plant information and the 3D plant scan files.
We found that Amazon Web Services (AWS) was able to best fulfill this role with AWS Cognito and AWS Identity and Access Management for user authentication and management, AWS DynamoDB to store plant information, and AWS S3 to store the 3D plant scan files. With the AWS Software Development Kit and Command Line Interface, we are able to connect our website with our Gaussian Splatting script to complete the pipeline.
