Using Eularian video magnification (EVM) to study the effect of fluctuations in relative humidity on cultural heritage materials.

Principal investigator:

Karen L. Pavelka

Project team:

Karen L. Pavelka, Senior Lecturer

[e-mail]
512-471-8269

Ayshea Khan, Master's Candidate

[e-mail]

Executive summary

In September 2015 our team began work to investigate whether Eularian Video Magnification (EVM) had practical applications for the conservation of cultural materials. Eulerian Video Magnification (EVM) is an open-source tool originally developed by MIT Computer Science and Artificial Intelligence Lab for medical monitoring environments. EVM can show movement that is not visible to the naked human eye such as a heartbeat within a chest. Our goal was to capture movement within objects as they shifted from one relative humidity (RH) to another.

The original plan for the research included a series of videos illustrating the effect of hysteresis on paper and photographs and creating a guide for others who might want to use EVM. Logistics precluded producing systematic videos, however we produced a website that features some of the videos we made, serves as a users manual for applying EVM to conservation, lists materials used, and will be updated periodically.

In June 2016 we presented the work we had done at Icon Conference 2016: Turn and Face the Change: Conservation in the 21st Century, Conference Aston, Birmingham (Turn and Face 2016). Shortly after we submitted the abstract Dr. Jonathan Kemp, Editor, Journal of the Institute of Conservation, asked us to submit an article to the Journal. We have submitted revisions from our first draft and are awaiting a decision on publication.

We found the EVM is a useful technique, and one that can be adopted with minimal cost or technical expertise. We will continue to explore the technique and publish our findings on our website (Magnified Movements n.d.).

Introduction

In September 2015 our team began work to investigate whether Eularian Video Magnification (EVM) had practical applications for the conservators working with cultural materials. Eulerian Video Magnification (EVM) is an open-source tool originally developed by MIT Computer Science and Artificial Intelligence Lab for medical monitoring environments. EVM can show movement that is not visible to the naked human eye such as a heartbeat within a chest. Paper and other materials are hygroscopic, expanding and contracting with fluctuations in relative humidity (RH). Although with rare exception we cannot see that movement, the evidence of it in the form of losses to media and cracks in primary supports is evident. Having a way to monitor subtle movements within objects and see specific points of vulnerability could provide a useful tool for preventive care.

We began the research with the intention of producing a set of videos showing how paper responded to changes in RH. We planned to begin with drastic changes and then use smaller and smaller increments as the work progressed. However, the humidity chambers we built ultimately did not allow the precision we needed in a reasonable timeframe. While producing a set of such videos remains a goal, the primary outcome of our work, and the real value of it, is 1) we illustrated that EVM can allow us to see movement of which we had been unaware and 2) we produced practical, cost efficient guidelines to use EVM.

Methods and Materials¹

Video studies

A Sony Handycam HDR video camcorder and a Nikon D300 were used to film the materials as they were subjected to variable relative humidity values. The Sony camcorder was used for initial videos as software and humidity chamber parameters were being determined. The Nikon, paired with a Phottix intervalometer, was used to create higher definition time lapses during the final testing period. This time-lapse workflow allowed for a longer observation period while minimizing the use of disk memory and processing time. Later tests were conducted with a newer Nikon D7200, which has a built-in intervalometer.

EVM processing

Wu et al. originally generated their 2012 EVM results through authoring a non-optimized MATLAB code (Wu 2012). MATLAB is a computing environment and programming language designed by MathWorks for use primarily within the fields of science and engineering. The application interfaces with other programming languages such as C++, Java, and Python. The code is open-source and can be downloaded freely and implemented within MATLAB for non-profit, educational research. Users are able to replicate the original results produced in the Wu et al. paper, or revise the code in accordance with specific test subjects.

Videoscope,² developed by Quanta Research Institute and CSAIL, is a web interface platform designed as an EVM proof of concept for the general public. Derived from the original EVM algorithms of the MATLAB implementation, the interface allowed for rapid trial and error testing of the magnification process at the beginning of this study. These initial tests gave promising results and helped form guidelines for the EVM amplification and frequency range parameters, recording workflow, and humidity chamber design. Longer video tests were later processed with the MATLAB program.

Materials tested

Glassine, kozo, lens tissue, handmade and machine made paper samples were subjected to various microclimates. RH was varied either by transferring the samples from one chamber to another; by changing silica gel canisters and sachets within a chamber, by adding or removing saturated or desiccated blotters within a chamber, by resting the paper on humid pieces of mat board, by spraying samples with a fine mist, or by applying ultrasonic vapor to the paper. Temperature was manipulated with incandescent light or heating pads.

Silver gelatin photographs, a wooden book board, pieces of linen cloth and strips of vellum were subjected to changes in RH by using blotters or silica gel canisters and sachets.

Humidity chambers

Through all of this work we have kept an eye on keeping costs low and methods simple. We used materials that are likely to be found in any conservation lab when possible.

The first humidity chamber was a simple tray, lined with wet blotters, plastic ceiling light panel sections, and nylon screening and covered with acrylic sheeting, but it did not work well. First, lighting was difficult in the small space. Second, the video camera did not seem to pick up movement through the acrylic sheeting.³ Another drawback of this chamber was the time required to change samples or humidity.

The second humidity chamber was built, or somewhat cobbled together, from a free standing, five tier plastic utility shelf. One section of the unit was modified with Davey board covered in polyethylene, Ethafoam, Plexiglas, Museum Glass, Mylar, magnetic strips and copious amounts of duct tape. It held a stable RH for at least 24 hours and allowed us to develop a working method.

The third humidity chamber is constructed of a frame of PVC plumbing piping, Ethafoam, Plexiglas, Museum Glass, Volara, foam insulation strips, Davey board, Mylar, magnetic strips, and thread. It is lightweight, portable and allows access from the top and side. It is easy to build and the cost of materials is low. Each of these chambers are described and pictured on our website (Magnified Movements, n.d.).

Lighting

Finding an affordable, reliable light source was difficult. A strong, raking light yields the best results. Battery powered sources are not reliable as the color temperature and intensity of the light fluctuate. The quartz photo flood lights⁴ from our photodocumentation set-up worked well, giving a bright, stable light and was useful for the short initial tests, but for longer periods they become much too hot. A 250 watt bulb in a porcelain socket, although cheap, had similar limitations. The LowelScandles Lights did not work well because the fluorescent light does not give sharp shadows. Fortunately, LED lighting continues to become more affordable. Although at the beginning of the grant period LED panels were beyond our budget, eleven months later we were able to purchase two LED panel kits.⁵ The color temperature can be adjusted between 3200K and 5600K and the lights are dimmable. These lights worked well, giving even, consistent light for more than an hour.

Results and discussion

EVM was designed to observe predictable and repetitive movements, but it can be adapted to be an effective tool to discover precise movements in inanimate objects. It allows us to view individual objects rather than generalize about a class of materials. The videos allow one to observe the specific point on an object where movement takes place. In some cases, as with very thin, responsive types of paper, the movement was observable in real time⁶, but denser materials required time-lapse photography⁷.

EVM employs three separate filters for magnification: Ideal for magnifying color variation, Butterworth for movement variation, and a low-order infinite impulse response (IIR) produces magnified results for both color and movement. The Butterworth filter revealed the most useful results for our purposes. The most effective frequency range for most magnifying videos of paper samples was .5 to .9 Hz, while an amplification value of 35 revealed the most movement without the introduction of interfering video signal noise.

EVM showed unexpected, and previously unseen, movement in paper when it was subjected to large fluctuations in RH. The paper samples were conditioned to a specific RH in a chamber for 24 hours or more and then the RH was adjusted up or down with saturated blotters, desiccated blotters or canisters of silica gel. The most striking observation is that movement in paper was completely unpredictable and erratic. Two samples cut from the same sheet of paper responded differently when subjected to the same tests and the same sample responded differently when a test is repeated. Some of the movement corresponded to the natural cockling that occurs in paper and was reflective of the wavy patterns seen in the margins of books from the hand-press period or prints that have not been matted. Other movements were completely unexpected, such as sudden pops that occurred at random positions and intervals on the paper’s surface. Generally, more movement can be detected in the outer margins of the sheets than in the center of any individual paper sample, and that movement appears to create pressure, causing the center of the sheet to deform; large bubbles appear and then deflate. The margins sometimes wave up and down; often the margin on one side would move and the other would remain flat. None of the movement we observed was predictable or consistent.

Conclusion⁸

EVM has proven to be a useful tool that can provide reliable information about the behavior of cultural heritage materials. It reveals specifically where and how the materials are moving, although the movement in the videos appears larger and more violent than what is actually occurring.

Our initial work with the EVM process has served as a preliminary exploration of its potential within the field of conservation. A goal of this study was to demonstrate how EVM can be used without access to an analytical conservation lab or a large source of funding, similar to the working circumstances many smaller institutions face. EVM is freely available, requires minimal tech support, and our humidity chamber was constructed with readymade and affordable materials. However, a more controllable testing environment could yield more precise information.

One avenue for research is to study the correlation between the density of materials and the frequency (Hz) at which they move. Discovering more about the specific frequencies of movement within materials would allow us to incorporate the PHASE implementation of the code, which is now recommended by MIT for improved motion magnification (Wadhwa, 2013). This program was developed in 2013 and utilized slightly different frequency parameters to yield a stronger magnification without an increase in video noise. Wadhwa et al.’s study on the movement of trees is particularly interesting, as they were able to perform a temporal frequency sweep that accounts for the movements occurring within the same object but at different temporal frequencies. Utilizing this feature of the code allowed for movement occurring within the tree’s trunk (lower frequency) and leaves (higher frequency) to be detected at the same time (Wadhwa 2013). A similar strategy could be applied with oversized or mixed media cultural heritage objects. This might provide a new way to monitor problematic works of art. For example, a camera could be focused on a composite work and it might be possible to see where the flaking and loss was about to occur. MIT has studied the imperceptible movements of construction cranes and trees using EVM, which suggests that there is no barrier to access in terms of objects’ scale (Wadhwa 2013). The software can therefore be a cost effective tool for those working in fields such as historic building, archeological, or sculpture conservation. We have posted, and will continue to post, the most significant of our videos on our website.

The evolution of 21st century technologies provides new avenues for innovative conservation strategy. Eulerian Video Magnification is an open-source tool that has the capability to complement existing microscopy analysis by revealing hidden movements within cultural heritage materials. This experimental study has produced surprising and erratic results within our paper samples as well as surprising movement within the wooden board. Further research and conversation with other information technology and conservation professionals could deepen our understanding of various materials’ movement as well as emerging video magnification technologies.

Notes

1. Most of the text in this section is taken from the paper submitted to Journal of the Institute of Conservation. It will be published either in the Journal or in the conference post prints.
2. The version of the Videoscope on which we did the original investigation has been replaced by a version with fewer variables although the web address remains the same, https://lambda.qrilab.com/site/. When we used the existing version of the Videoscope we did not get the same results we had gotten earlier.
3. A series of test videos comparing various types of glazing showed that Tru Vue Museum Glass gave the clearest results for filming. The most significant images from those tests are on our website (Magnified Movements n.d.).
4. These lights were acquired in approximately 1980 and there is no label or identifying mark on them.
5. Fotodiox LED panels, Model LED312DS. The pricing on these lights seems to fluctuate, but we purchased them for approximately $100 each.
6. See Glassine 06 and Glassine 07 on Magnified Movement website; http://www.magnifiedmovements.com/video-library/
7. See Wood Leather 02 on Magnified Movement website; http://www.magnifiedmovements.com/video-library/
8. This conclusion is paraphrased from the paper submitted to Journal of the Institute of Conservation. It will be published either there or in the conference post prints.

Acknowledgements

We would like to thank the National Center for Preservation Technology and Training, a unit of the National Park Service, who generously funded this research, as well as the many iSchool students and colleagues at the School of Information who have assisted us throughout the course of this project.

Bibliography

• Magnified movements: using Eularian video magnification with cultural heritage objects. http://www.magnifiedmovements.com.
• Pavelka, Karen L., and Ayshea Khan. "Using Eularian video magnification to discover movement in objects." Turn and Face the Change: Conservation in the 21st Century, 2016. http://icon.org.uk/system/files/public/Conference/icon-conference-programme-final-5.pdf.
• "Project Description." Magnified movements: using Eularian video magnification with culturall heritage objecta. http://www.magnifiedmovements.com/project-narrative.
• Quanta Research Institute. Small change. Big difference. Last modified 2015. https://lambda.qrilab.com/site/.
• Sabera, Donald K. Isoperms: an environmental management tool. Washington, DC: Commission on Preservation and Access, n.d.
• Strang, Tom, and David Grattan. "Temperature and humidity considerations for the preservation of organic collections – the isoperm revisited." e-Preservation Science, no. 6 (2009): 68-74.
• Turn and Face the Change: Conservation in the 21st Century. London, England: ICON, 2016. Accessed December 15, 2016. http://icon.org.uk/system/files/public/Conference/icon-conference-programme-final-5.pdf.
• "Understanding temperature and humidity for books." Climate Notes Newsletter. http:///imagepermanenceinstitute.org/resources/newsletter-archive/v27.
• "Video library." Eularian video magnification: using Eularian video magnification with cultural heritage objects. http://www.magnifiedmovements.com/video-library.
• Wadhwa, Neal, Michael Rubinstein, Fredo Durand, and William Freeman. "Phase-based video motion processing." ACM Transactions on Graphics 32, no. 4: 80:1-80:9.
• Wu, Hao-Yu, Michael Rubinstein, Eugene Shih, John Guttag, Durand Fredo, and William Freeman. "Eulerian video magnification for revealing subtle changes in the world." Transactions on Graphics 31, no. 4 (2012): 1-8. doi:10.1145/2185520.2185561.

Further References

• Banick, Gerrard, and Irene Bruckle. Paper and water: a guide for conservators. Oxford: Taylor & Francis Ltd, 2011.
• Bickersteth, Julian. "Environmental conditions for safeguarding collections: what should our set points be?" Studies in Conservation 59, no. 4 (2014): 218-24.
• Chambino, Pedro Botolo. "Android-based implementation of Eulerian video magnification for vital signs monitoring." Faculdade De Engenharia da Universidade do Porto, 2013, 1-53.
• Kirby Atkinson, Jo. "Environmental conditions for the safeguarding of collections: a background to the current debate on the control of relative humidity and temperature." Studies in Conservation 59, no. 4 (2014): 205-12.
• Lennard, Frances, and Janice M. Dulieu-Barton. "Quantifying and visualizing change: strain monitoring of tapestries with digital image correlation." Studies in Conservation 59, no. 4 (2014): 241-55. doi:10.1179/2047058413Y.0000000097.
• Luxford, Naomi, and David Thickett. "Monitoring complex objects in real display environments–how helpful is it?" Climate for Collections-Standards and Uncertainties. Postprints of the Munich Climate Conference 7 to 9 November 2012, 2013, 255-70.
• Michalski, S. "Double the life for each five-degree drop, more than double the life for each halving of relative humidity." 13th triennial meeting of the ICON Committee for Conservation, Rio de Janeiro, 2002.
• Staniforth, Sarah. "Environmental conditions for the safeguarding of collections: future trends." Studies in Conservation 59, no. 4: 213-17.
• "Tapestry Monitoring Research Project." University of Southampton. http://www.tapestry-strain.org.uk/tapestry-monitoring-research-project.
• "Understanding temperature and humidity for books." Climate Notes Newsletter, March 2014. https://imagepermanenceinstitute.org/resources/newsletter-archive/v27/understanding-temperature-and-humidity-books.