One way to ‘collect’ a massive specimen: Simple photogrammetry in the field using a mobile phone
Nigel Larkin and Steven Dey (UK)
Inspired by the excellent series of articles by Trevor Watts discussing the types of Mid-Jurassic dinosaur footprints to be found along the Whitby coast (see The dinosaur footprints of Whitby: Part 1, for Part 1 – links to the other parts can be found at the end of that part), when recently working in the area I (NL) made sure that I would have the time to walk the beaches from Saltwick Bay to Whitby. I also timed my work to make sure I could make use of the low tides early in the morning at first light. As well as the usual ammonites, belemnites and plant fossils, I found a handful of single footprint casts (most too heavy to attempt to move) and some very nice fallen slabs of claw marks and partial trackways – also mostly too big to move.
One slab in particular stood out among the others at the bottom of the Ironstone Ramp in Long Bight (Figs. 1 and 2) – a ‘double trackway’ from what look like two quite different beasts walking in parallel – although they were possibly formed at different times. In the form of raised footprint casts rather than actual indented footprints, the specimen included five good prints in the left track and four, possibly five prints, on the right track – so each track contained a ‘full set’. Although the tracks look superficially quite different from one another, both appear to be attributable to theropod dinosaurs.
The slab was not buried in the sand, but sat on some other fallen blocks so could be seen in its entirety. However, at about 1.2m x 1.0m x 0.2m, it must have weighed in excess of 500kg. To ‘collect’ it, one option was to break it up and take it away in pieces in several visits and put it back together again at a later date, but that would have been less than ideal to say the least and small fragments might have been lost. A preferable option would have been to carry the complete slab all the way back to Whitby in one piece, requiring a group of people walking along the beach at low tide.
The first five metres or so would have been a difficult scramble over all the other fallen blocks while carrying the heavy weight, but it would have been possible to do this relatively safely by using beams of wood to lever the slab carefully onto a strong pallet, then sliding the beams inside the pallet and carrying it away ‘stretcher’ fashion – if enough people and time had been available. However, ‘Storm Doris’ was about to hit and the chances were that the specimen would be broken into pieces by the next day. Even if it survived the storm, it was possible that, although most people finding it would admire the specimen, maybe take photos and then leave it where it was, someone else might take a hammer and chisel to it and try to remove the nicer of the raised footprints on the left side of the block.
The best option – which did not preclude returning at a later date with a gang of people to collect the specimen for the local museum if it survived the storm – was to take as many photos as possible there and then to record the specimen. This would involve not just taking the pictures with something included for scale (as well as the obligatory ‘selfie’) but taking photos from as many angles as possible from approximately the same distance, including around the back and underneath as far as possible, specifically so that a photogrammetric 3D digital model could be made of the specimen at a later date.
The problem was that the only camera I had on me was a rather old ( about 2013?) and cheapish mobile phone with limited memory. So I took 65 photos with this. I returned the next day to find that the specimen had thankfully survived Storm Doris and I took 81 more photos, this time with a proper digital camera (a Canon Powershot SX50 HS).
Not only did this mean that the double trackway was well recorded before it got damaged, but it also meant that two digital photogrammetric models could be built, one from each set of images. It would also be interesting to compare the two models to see if the old mobile phone camera would provide something useful or whether this sort of photogrammetric work can only be achieved with a ‘proper’ digital camera. To ensure the best chance of good digital models being made from these images, I sent the photos to Steven Dey, a photogrammetry, laser-scanning and 3D-printing expert at ThinkSee3D based in Oxfordshire, with whom I have worked on many palaeontological projects before. Below, he describes the processes of making the 3D digital models, as well as giving top tips for taking good photogrammetry photos in the field.
Photogrammetry (also known as SfM – Structure from Motion) generates digital three-dimensional models from multiple photographs taken from different positions and angles around an object. An advantage of photogrammetry compared with other methods, such as laser scanning, is that the ‘scanning’ part of the process is very cost effective as all that is needed is a digital camera to capture the data and even a mobile phone camera will do. This means it is an ideal method for use in the field, when out looking at large specimens and geological features.
It is even possible to take accurate measurements of the resulting digital 3D specimen and to measure features, particularly useful if the specimen itself was difficult to access. To ensure this, place a scale bar or an object of known size (a geological hammer, hand lens or trowel for example) on the specimen in a few photographs to act as a reference and this will allow the digital 3D model to be made with a relevant scale. You can also put a colour scale in the scene, so the colours of the phototexture can be colour balanced if you want accurate colour reproduction.
Informal comparative trials have shown that the accuracy of virtual 3D models of fossils produced with photogrammetry can be equal to or even better than other scanning methods, but this accuracy is dependent on four main influences: (1) the quality of the photography; (2) the quality of the light in the environment; (3) the surface quality of the specimen; and (4) the processing of the images into a 3D model.
1. The quality of the photography
Most considerations are common to all photography, such as reducing camera shake (a tripod can help), good focus and an appropriate exposure to minimise over-exposed or under-exposed areas. However, some considerations are particularly relevant to the photogrammetry process, such as ‘depth of field’. Oblique angled photographs across a specimen, especially a long object, can be partly out of focus due to a limited depth of field if care is not exercised, so avoid overly oblique shots. ‘Shutter priority’ can be a useful setting on a digital camera for photogrammetry, as this allows the ‘f’ number to be tuned to maximise the depth of field.
Distance from the subject is also worth considering. Normally, with a standard 50mm lens, a distance of around 0.5m or so works well. Too close and lens distortions can cause issues. Too far away and the resolution of the images – and therefore the resolution of the eventual model – will be reduced. The resolution of the digital model is directly related to the distance from the subject, and to the size and resolution of the image sensor in the camera. An ‘auto’ setting on the camera can normally be an appropriate facility to use, but it can also be a problem if very different brightness levels are experienced in different areas of the specimen, for example, sharp differences in light and shade on a sunny day. Rarely are conditions ideal in the field, so it is worth trying a few configurations of camera settings if there is time and, if you have such options, taking test shots from different angles and reviewing them before taking the final set of images all on the same setting.
A definite departure from normal photography is the need to capture multiple photos from numerous angles around the subject. Every angle of the specimen that needs to be scanned has to be captured with an 80% overlap between photos. Fig. 3 shows the various positions (the blue rectangles) of the mobile phone used to model the trackway slab on the coast at Whitby. To have captured the whole of the specimen, it would have been necessary to scan one side first, then turn it over and scan the underside, but this could not safely be undertaken with such a heavy slab.
As an example, however, this was easily done for the 32kg single sauropod footprint cast collected on the same day using just 45 photos (Fig. 4).
2. Light quality
The perfect conditions for taking photographs for photogrammetry purposes are bright diffuse light, such as being outside on a cloudy day in the spring or summer, as changing light, dark shadows and/or bright sunlight on surfaces are not ideal. Highlights from bright sunlight tend to move position between different photographs, and shadows obscure the surface colours and texture of the specimen. Both of these would lead to ‘noise’ and inaccuracies in the final 3D model. If indoors, use diffused daylight from windows or artificial ‘daylight’ lighting, or set the camera to compensate for electric lighting.
3. Surface quality
Photogrammetry works best with highly textured subjects. It does not work on very shiny objects, or transparent or very monochrome objects. For example, a very plain white wall would not scan, as the algorithms in photogrammetry need to see features ‘moving’ from image to image to determine depth. If those features do not exist or reflective highlights move between shots, the process of building the virtual model will fail. The highly-textured surface of fossils, such as footprints and their surrounding rock is particularly suited to photogrammetry, as the algorithms involved rely on differences in surface colouration and texture to recreate the 3D geometry.
4. Processing the images
Processing the data efficiently requires a computer with a reasonably high specification, such as my (SD) Intel i7 with 32Mb RAM and a good GPU card (for example, NVIDIA). Even then, processing photographs into 3D models is an extremely computing-intensive process, often requiring many hundreds of millions of calculations. So, if there are a lot of photos, it is sometimes necessary to reduce the resolution. Photogrammetry software can help by allowing the data to be broken into blocks to process it in smaller sets and thereby not overwhelming the computer’s resources all in one go. The software we used was Agisoft Photoscan standard edition.
Weaknesses in any of the above four factors can lead to ‘noise’ in the final virtual model or lower resolution of detail, or, in extreme cases, failure to create the 3D model. It is usual for some of these factors to be imperfect, so compromises have to be reached, but the photogrammetry process itself is quite robust and can sometimes cope with quite poor inputs. In reasonable conditions, much larger specimens than the footprint slab can be captured this way, even large geological features and indeed whole landscapes.
Virtual 3D models can even be made from photographs taken underwater, taken by drones or other aerial photography and even microscopic photogrammetry is possible – as long as enough good quality photographs are taken in good conditions. But 3D model processing considerations have to be accounted for. It might take many hours to process a very large set of photographs into a 3D model, depending on the specification of the computer.
The whole scene of the Whitby footprint slab as a 3D model without the colour overlay is shown in Fig. 5. The output of the process is effectively a 3D photograph of the scene. The texture file overlays multiple photo images on to the 3D geometry adding data on the surface colouration. Looking at the model without the colour overlay can sometimes reveal morphological features that are otherwise hard to distinguish.
Three digital models were created of the slab containing the trackways at Whitby. The first model (Fig. 6) was made from photos taken with the old mobile phone (65 images on a GT-I8190N camera on a 2014 Samsung Galaxy phone). The second model was made from photos taken with a compact SLR camera (81 images on a Canon Powershot SX50 HS 4.3). A third model was made by combining these two sets of data. In all cases, the diffuse and constant light on the specimen from naturally bright but cloudy conditions was ideal.
The images were loaded into Agisoft PhotoScan and rendered into 3D models following the workflow in the application: aligning photos, building dense point cloud, building mesh, and building texture. All processes were set to high quality and decimation in the mesh build was avoided by using a custom mesh size set to a high value. PhotoScan performs camera calibration automatically using Brown’s distortion model, but the photos had some EXIF data, which assists the calibration and camera alignment process.
The texture file was a jpeg set at 4,096 x 4,096, using Photoscan’s generic texturing method. The finished 3D models were exported from Photoscan as an OBJ file, with associated material and photo-texture file, and uploaded to the online 3D viewing and sharing platform, Sketchfab, for sharing and discussion (see links below).
Clarity of both models was found to be of high quality, showing millimetre scale details of the trackways in the 3D geometry and in the attached texture file. The digital SLR camera would usually produce a better result than the mobile phone camera, because the quality of the optics and resolution of the sensor means it can capture more information. However, the latest cameras in mobile ‘smartphones’ are really very advanced and can make excellent photogrammetry tools. In poor light conditions, they can even produce better results than mid-range SLR digital cameras. They also have the advantage of being much simpler to use.
In the case of the Whitby trackway, photos taken with the old mobile phone produced very nearly as good a result as the digital camera and the model was certainly more than adequate as a record, and for identification and measurement purposes.
In conclusion, capturing multiple images of an otherwise uncollectable object in the field with an old mobile phone camera can enable a scientifically useful 3D digital model to be built to record the specimen and from which measurements can be taken. This data can even be used to 3D-print a displayable solid replica of the fossil, if required. Mobile phone cameras are improving every year so results using this technique are going to get even better.
The digital models discussed can be found at these addresses, but they might be slow to open:
The model made from mobile phone photos can viewed here:
The model made from the digital SLR camera photos can be viewed here:
The photogrammetry process was also undertaken for the single sauropod footprint cast, using just 45 photos to capture the entire specimen, including the underside (Fig 5). This model can be viewed here:
About the authors
Nigel Larkin is a freelance palaeontological conservator, preparator and curator, whose research interests include trace fossils and ichthyosaurs. Steven Dey runs ThinkSee3D, a company specialising in digital and physical 3D realisation projects for research and public engagement (including 3D printing, AR and 3D scanning).