A Writers Notebook
A mixed reality media installation using vision/sensor fusion
A Writers Notebook, shown in the in-conjunction exhibition to the International Symposium on Electronics Arts in July 2008. The installation uses research in real-time stable marker-less tracking techniques developed at the Interaction and Entertainment Research Center.
This first installment places a virtual characterization of the writer Somerset Maugham in proximity with the viewer in such way that allows the viewer to move around and view from multiple angles. A video describing the mixed treality technology and installation is below:
Participants wearing head mounted display systems, witness virtual characters of various notable figures, including Somerset Maugham, Joseph Conrad, and Jean Harlow, immersed within a real world environment that is modeled on the Raffles Hotel Long Bar they once frequented. This system allows the user to experience pseudo-historical events impressed over a present day
real world environment. Through the application of research in occlusion, and by embedding large mesh animated characters, this installation demonstrates the results of the technical research and the conceptual development of immersive multimodal interaction. A Writers Notebook is a "sketch" in the development of the larger mixed reality installation, "The Long Bar." The Long Bar is an augmented and mixed reality multi-media art installation set at the famous Long Bar at the Raffles Hotel. The installation uses AR and MXR technologies to develop historical, culturally significant events into a multi-modal immersive interactive experience.
This installation in the ISEA exhibition places a virtual character representing the writer, Sommerset Maugham in proximity with the viewer in such way that allows the viewer to move around and view from multiple angles while the character recite fragments of narratives drawn from Maugham's work.
2009 Pensyl, W. (Russell),), Tran, C.T.Q., Lee, S.P, Jernigan, D. K. Robust Hybrid Tracking with Life-size Avatar in Mixed Reality Environment, 13th International Conference on Human-Computer Interaction 2009,San Diego, California, USA
1. ARTOOLKIT http://www.hitl.
washington.edu/artoolkit
2. CAARLS, J., JONKER, P.P., PERSA, S. Sensor Fusionfor Augmented Reality. EUSAI 2003: 160-176
3. HOL, J.D. SCHON, T.B., GUSTAFSSON, F., SLYCKE, P.J. 2006. Sensor fusion for augmented reality. In Proceedings of 9th International Conference on Information Fusion.
4. KATO, H., BILLING-HURST, M. 1999. Marker tracking and HMD calibration for a video-based augmented reality conferencing system. In
Proceedings of the 2nd International Workshop on Augmented Reality (IWAR 99). October, San
Francisco, USA
5. KSATRIA GAMEWORKS. http://www.ksatria.com
MXRTOOLKIT http://sourceforge.net/projects/mxrtoolkit
6. NGUYEN, T.H.D., QUI, T.C.T., XU, K., CHEOK, A.D.,
TEO, S.L., ZHOU, Z.Y., ALLAWAARACHCHI, A., LEE, S.P., LIU, W., TEO, H.S., THANG, L.N., LI, Y., and
KATO, H. 2005. Real Time 3D Human Capture System for Mixed-Reality Art and Entertainment, IEEE Transaction
on Visualization and Computer Graphics (TVCG), 11, 6 (2005), 706 - 721.
8. SLABAUGH, G., SCHAFER, R., LEVINGSTON, M. 2001.
Optimal ray intersection for computing 3D points from Nview correspondences.
9. SONG, M., ELIAS, T., MARTINOVIC, I., MUELLERWITTIG,
W., CHAN, T.K.Y., 2004. Digital Heritage
Application as an Edutainment Tool. In Proceedings of the
2004 ACM SIGGRAPH International Conference on Virtual Reality Continuum and its Applications in Industry
10. SPARACINO, F., WREN, C., DAVENPORT, G., PENTLAND, A. Augmented performance in dance and theater. In International Dance and Technology 99, ASU, Tempe, Arizona, February 1999.
11. UEMATSU, Y., SAITO, H. 2007. Improvement of accuracy for 2D marker-based tracking using particle filter. In Proceedings of 17th International Conference on Artificial Reality and Telexistence.
12. WII REMOTE http://wiibrew.org/wiki/
Wii_Remote
13. WII YOURSELF. http://wiiyourself.gl.tter.org/