Virtual Humanities 1: Archive Project Launch

ReCap Photo example

Immersive environments are one of the newest areas of DH experimentation. 3D simulation, Virtual Reality and Augmented Reality were until recently only accessible with high end hardware and produced end products that were so large they were hard to deliver over the internet. As the technology improves, internet bandwidth grows, and the computers in our pockets get more and more sophisticated, these projects are now coming into their own.

In the past week, you’ve gotten a brief introduction to digital collections and metadata through the collection management platform Omeka and our Digital Objects of Carleton sample site featuring archive objects. Today, we are taking a deeper dive into the Carleton College Archives and its shiny new database to help work on a class project 3D modeling and exhibiting objects from the archives. This class project will both help jumpstart the archives’ effort to create a 3D gallery of their collections and serve as an example of the type of project you might want to do for your final project.

The Carleton College Archives new database that supports digital objects including 3D models

3D Collections and Photogrammetry

There has been a large initiative among museums around the world to create 3D versions of objects in their collections. Among these include the British Museum, which has made hundreds of scans publicly viewable on the popular 3D modeling site SketchFab, and the Smithsonian, which has designed several online 3D exhibits (including this creepy one of Abraham Lincoln’s death mask!).

How to Make a Model

Manual 3D modeling techniques like those we’ve attempted in Fusion360 are very effective and have had a long history of producing impressive digital humanities projects.  Lisa Snyder’s long-running project to recreate the World’s Columbian Exposition of 1863 in Chicago is a prime example of what these techniques can accomplish in skilled hands.

1863 World's Columbian Exposition

Increasingly, however, computers are doing more of the heavy lifting. There are several methods of generating 3D models that rely on algorithms to create geometric meshes that are being adopted for DH projects along with techniques borrowed from the film and video game industries like motion capture. The award winning Virtual Angkor being a prime example of the leading edge of these techniques.

https://www.virtualangkor.com

Two major current methods for generating 3D content are procedural modeling and photogrammetry, which we’ll discuss today.


Procedural Modeling

This term refers to the generation of complex geometry from basic shapes through the application of code-based rules.  The leading platform for this type of work in DH is CityEngine, owned by ESRI, the makers of ArcGIS.  This technique allows a user to produce, modify and update large, textured models of entire cities quickly and iteratively.  The output can be explored online or integrated with gaming software or 3D animation packages to produce video games, simulations and movies.

This software was developed for modern city planners and urban architects, but has increasingly been put to use on historic landscapes and built environments, as in the impressive work of Marie Saldaña who developed a Roman temple rule set.

CE workflow, Marie Saldana

For the past several years, a project here at Carleton led by Serena Zabin and Austin Mason has been using procedural modeling to recreate colonial Boston circa 1770 as part of a video game centered on the Boston Massacre called Witness to the Revolution. You can see a flyover of part of the reconstruction in the video below and read more about the origins and process from Carleton students’ posts on the development blog.

Boston central wharf area reconstructed with procedural modeling

Procedural modeling is very powerful, but we’ve taken the game in a different direction with professional 3D artist support to hand model the city in a sytlized art style, as can be seen on the project website.

Trailer for Witness to the Revolution, 2021

Photogrammetry

Photogrammetry is another algorithmic modeling technique that consists of taking multiple overlapping photographs and deriving measurements from them to create 3D models of objects or scenes.  The basic principle is quite similar to the way many cameras these days allow you to create a panorama by stitching together overlapping photographs into a 2D mosaic.  Photogrammetry takes the concept one step further by using the position of the camera as it moves through 3D space to estimate X, Y and Z coordinates for each pixel of the original image; for that it is also known as structure from motion or SfM.

Photogrammetry can be used to make highly accurate and realistically photo textured models of buildings, archaeological sites, landscapes (if the images are taken from the air) and objects.  Close range photogrammetry of historical objects offers the possibility of both digitally preserving artifacts before they may be lost or damaged, and of allowing a whole suite of digital measurements, manipulations and other analyses to be performed that allow insights into the material that might not be visible to the naked eye.  The technique is gaining in popularity and usage, since it produces very impressive results comparable to high end laser scanning technologies for a mere fraction of the cost.

In-Class Exercise: Choose your own Photogrammetry Adventure

There are an increasing number of methods for creating photogrammetry models in the market, and we are going to experiment with several of them and compare the results. Some of these methods require dedicated local hardware with lots of RAM and advanced graphics processing capabilities, others are cloud-based which offer speed and ease of use at the cost of some control over your data and the details of the process.

We will explore the following methods for creating photogrammetric models:

  1. Agisoft MetaShape (the venerable desktop-based solution to process photos)
  2. Autodesk ReCap Photo (a cloud-based solution run from the desktop that integrates with other Autodesk offerings)
  3. Scaniverse (an app that leverages the latest camera and LiDAR scanner technology of iOS devices)
  4. Polycam (an app that works on any phone and can use photos or LiDAR to process and edit models)

You will form groups around an object, and each member of the group will try a different method. Some of these are quicker than others, and we will compare and clean up the results in class next week to share the best models with the Archives.

1) Taking Photos

  • 1 team member will take photos with their phone camera
  • The others will use the lights on their phones to cast side lighting on the object — DO NOT MOVE
  • These photos will be used for desktop applications and data recording for future proofing

The most important step in photogrammetry is collecting data: i.e. taking good photographs that will have sufficient information for the algorithms to stitch images together and extract 3D geometry from them.

Tips on taking photos for photogrammetry — pay attention to the first section on a fixed object

With a mobile phone camera, you’ll want to make sure you follow these guidelines:

  • Have your scene well and consistently lit — DO NOT CHANGE LIGHTING between shots.
  • Turn off the flash — because, again, DO NOT CHANGE LIGHTING between shots
  • Use the standard photo settings rather than a “portrait” mode or similar. You want deep focus
    • If Macro mode turns on, turn it off
  • Have the object fill as much of the frame as you can
  • Start at one point and circle the object, overlapping photos by ~50%
  • Repeat at multiple heights to get full coverage
  • Take several close up shots to get shadowed areas and detailed textures — but DO NOT CHANGE camera settings, zoom, etc.
  • If possible, get shots of the top and bottom of the object for full 360 degree coverage

2) Using Phone Apps

  • 1 team member will download and install one of the apps, and follow the tutorials linked below
  • The others will use the lights on their phones to cast side lighting on the object — DO NOT MOVE
  • These apps will process the models live and allow you to share and export results almost immediately

3) Processing Photos (can be completed in class Tuesday)

MetaShape (Mac or Windows)

Agisoft screen shot

A demo mode of MetaShape is available for free that will let you try everything except exporting and saving your model. If you want to explore more, they offer a 30-day free trial of the full Standard or full Professional editions, but the software is very expensive after that.

Processing Photos

The main steps in the MetaShape workflow are:

  1. Add photos
  2. Align Photos
  3. Build Dense Cloud
  4. Build Mesh
  5. Build Tetxure
  6. Inspect and clean
  7. Export to desired format(s)

Follow the instructions in the tutorial below

For more, Agisoft has excellent tutorials on their website that stay up to date with advances in the software. Here is one for how to build a complete (360-degree) model.


Autodesk ReCap Pro (Windows)

Autodesk ReCap Pro loading screen

ReCap Photo is a good solution for Windows, as a cloud-based photogrammetry software that is part of the Autodesk suite (along with Fusion360), but can be finicky to install.

  • Make sure you are logged into Windows on your lab computer (the software does not work on MacOS)
  • Go to the Autodesk Education site, login to your account, and Download ReCap Pro software.
  • Make sure the checkbox is checked to also install Recap Photo (see above)
  • Launch the Autodesk ReCap Photo software from the desktop (NOT ReCap)
    • Sign in top right if not already logged in

Processing Photos

ReCap Photo makes processing easy by handling most of it for you in the cloud. The main steps are:

  1. Load photos
  2. Upload to cloud
  3. Wait for processing
  4. Download model
  5. Inspect and clean
  6. Export to desired format(s)

Follow the instructions in the tutorial linked below:

Lab Assignment (due Sunday):

Your assignment for this week is to make at least one 3D model of your object and add it to our Omeka repository

SELECT YOUR OBJECT

  • During class, you will select an object to focus on in a group of 2-3 students.
    • Note: Have one student focus on taking/uploading photos and the others on creating the Omeka entry and researching the object.
  • During or after class, go to our list of archive objects on the google drive and enter the names of your group members next to your chosen object.

PHOTOGRAPH YOUR OBJECT

  • During class, take 30-80 photographs of the object by walking completely around at least twice, taking photographs at multiple heights/angles + top coverage and bottom, as much as the table allows.
    • Don’t forget some close up shots of texture details and especially any shadowed areas.
      (NB: take closeups by CAREFULLY moving physically closer — NOT by adjusting the zoom on your camera).
  • Upload your photos to an appropriately titled folder in the Archive Objects shared Google drive
    • NB: If your phone took the images as HEIC files, you’ll need to convert HEIC to jpg
    • I also recommend uploading your photos with a cord connection to your computer. Copy the photo files from your phone, to a local desktop folder, then upload to Google Docs.

Make a 3D model using photogrammetry

  • Use at least three of the methods above
  • Export your model to get 3 different versions to save in your Google Drive folder
    • An OBJ file (for sharing) by following tutorial instructions
    • A GLB file (for online viewing).
      • For this, you may need to convert the OBJ file into a gLTF/gLTB file in Blender, following the tutorial instructions
  • Save project files and all exports to your team’s folder in the Common > ArchiveObjects google drive

Write a joint lab report on your process so far

Collectively, reflect on your experiences with the Archive project so far in light of the readings and examples of photogrammetry and cultural heritage modeling you’ve explored.

  • Include an image of your object
  • A brief description
  • Embed an interactive view of the model from an app
  • Answer at least two of the questions below
    • How does attempting to model an object compare to simply viewing one in an exhibition?
    • Does the process of photogrammetry encourage close looking and attention to details you might otherwise have ignored?
    • Does engaging with objects in this way make you more curious about this (and/or other) things, as well as the stories they can tell?
    • How does moving from passive observer to active modeler change your thinking on communicating about art and history to various publics?

Turn in your assignment

  • Finally, each group member should individually submit the link to your one post by pasting in the Moodle assignment

Specifications for Sunday’s assignment

Complete Labs must include

  • In Google Drive Folder
    • All photos taken of object in Google Drive folder
    • 3D model file and exports in Google Drive folder
      • .obj
      • .glb
  • WordPress Post
    • Link to your group’s one collective lab report that includes
      • an image of your object
      • A brief description
      • An embedded interactive view of the model from an app (e.g. SketchFab model viewer)
      • Answers to at least two of the reflection questions listed above

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

css.php