Rendered Database

[Goal] Make Rendered Database using Rendered Image Obtained by Trained 3D Gaussian Splatting

Get Rendered Image using 3D Gaussian Splatting Rasterization

• Benchmark Dataset → Tanks and Temples

Render Img1 (Novel View Synthesis)	Render Img2 (Novel View Synthesis)	Render Img3 (Novel View Synthesis)

---

Prerequisites

1. 3D Gaussian Splatting
2. NetVLAD Pytorch
3. Faiss
4. Anaconda
5. OpenCV with Contrib
6. RoMa
7. LightGlue
8. SuperPoint
9. Feature Matching

---

Download pre-trained model

Download pre-trained model

• https://drive.google.com/drive/folders/1vhsYBUaxEPKSiBWgKTnHsH700aiTtPjA?usp=sharing

---

Preparing Dataset (in example folder)

(1) cameras.txt → [IMAGE_ID, CAMERA TYPE, IMAGE Width, IMAGE HEIGHT, ~]
(2) images.txt → [IMAGE_ID, QW, QX, QY, QZ, TX, TY, TZ, CAMERA_ID, NAME]
(3) image folder

---

Combined Preparing Dataset (Make KeyFrame DB)

• Using NetVLAD Method to make global descriptor

  • KeyFrame = {Index, Image, Keypoints, Descriptor(local or global), Camera Pose, Camera Type, Camera Params}

• Final Database Extension file is HDF5
  

---

Method of Keypoint and Descriptor (in config.yaml)

• extractor_method (extractor_utils.py):

  • 0 → apply ORB
  • 1 → apply SIFT
  • 2 → apply AKAZE
  • 3 → apply SuperPoint Model
  • roma_based_extractor function → apply RoMa Model

• descriptor_method (extractor_utils.py):

  • 0 → apply ORB
  • 1 → apply SURF
  • 2 → apply DAISY
  • 3 → apply AKAZE
  • 4 → apply SuperPoint Model

• matcher_mode (matching_utils.py):

  • calculate_score function → apply BF Matcher (Brute-Force) + KNN Matches
  • roma_based_extractor function → apply RoMa Model
  • lightglue_matcher function → apply LightGlue Model

• REJECTION_MODE (using cv2):

  • apply cv2.fundamentalMatrix

---

Example Results

• (1) Find the Nearest Distance using Faiss obtained by VLAD vector (Find Candidates)

• (2) Find Feature Matching using Local Descriptor (Finda the Best Candidates)

  Matching Results (ORB + DAISY + distance=0.99)

  Matching Results (RoMa indoor + reproj=0.01)