//
//	VISUALIZATION BASIC 
//


// ===== Opening title =========================

(speed of speech: 1.0)
(opening image: vizdata/title-2.png)
(opening music: vizdata/MissionRank3.wav)
(opening title: 6.0 seconds)

(intro start)
Hello, I'm Pooni, working for Masaki Hayashi.
(intro end)

So, let's start.

(sound fadeout)

// ===== Lecture start =========================

//
//	Visualization basic - title
//
(image:vizdata/viz-2/Slide1.PNG)

Hello everybody.

This is the first lecture of the course. I′m going to talk about the very basic of visualization technology.

//
//	Visualization basic - 2
//
(image:vizdata/viz-2/Slide2.PNG)

The purpose of today′s lecture is to give you an overview of visualization technology and to show you some videos to understand the available technologies which can be used for visualization. 

Actually, the word Visualization has a very broad meaning. If you look into the Wikipedia, for example, you will see many kinds of visualization. In this course, I will mostly focus on visualization methods using images and movies and 3D computer graphics so that you can present your things to other people.

(pause:2)

In today′s lecture, I will talk about some visualization technologies, virtual museums and finally a little bit of network-based services.  

//
//	Visualization basic - 3
//
(image:vizdata/viz-2/Slide3.PNG)

First, the visualization technologies.

//
//	Visualization basic - 4
//
(image:vizdata/viz-2/Slide4.PNG)

I′m going to start with the general knowledge.

Here you can see three phases in digital media with visualization. Let′s assume that you have something that you want to show to the people in the public through visual media.

For that purpose, first you need to digitize things that you have, and to get the digital data of the thing, then you show the digital data in any kind of media to the people, and if your media is connected to the internet you will disseminate the content toward all over the world.

So, there are three phases, those are Digitization and Representation and Dissemination.

In this course, some workshops in the last half will deal with digitization such as filming flat paintings, taking photographs and also scanning 3D objects. 

Today, I will mostly talk about representation. I mean, if you have something to show to the people, what kind of way would be possible and available for you to do that. There are actually many ways to represent your data.

In this lecture, I will provide you various available methods.

(pause:2)

And, finally, I will talk a little bit about the dissemination. Just a little bit. What special thing would be possible when you use collective knowledge on the Internet.

//
//	Visualization basic - 5
//
(image:vizdata/viz-2/Slide5.PNG)

When you want to design your visualization system or service, what aspects do you need to consider?

Here′s a hint. There′re three aspects, those are devices, visual representations, and dissemination methods.

The devices could be, PC, monitor, screen projection, handheld device, head-mounted-display and so on. And for the visual representation, you may use 2D picture, 2D movie, 2D panoramic image, 3D CG, stereoscopic, hologram and so on. The hologram is a bit extreme method, though.

And finally, when you want to disseminate the content, you could use public space, mobile, Web, electric book and so on.

So, actually, there are many ways to do that.

//
//	Visualization basic - 6
//
(image:vizdata/viz-2/Slide6.PNG)

So, eventually, something like this.

There are three aspects, devices, representations and dissemination, and each category has several choices that you can decide to use.

For example, You may decide to use a handheld device with stereoscopic images and distribute it in the public space and mobile environment. Yes, something like that.

So, in order to design the service. I mean if you have something to show to the public, it would be better for you to know the available technologies to fulfill your purpose. And that′s what I want to show you in this lecture.

//
//	Visualization basic - 7
//
(image:vizdata/viz-2/Slide7.PNG)

So, I′m going to show you the technology showcase.

//
//	Visualization basic - 8
//
(image:vizdata/viz-2/Slide8.PNG)

The first one is the straightforward way to show images to people, which is a monitor or you can say, a display.

I think you have already known a 4K resolution monitor. Or you may even have purchased a 4K television in your home! Well, yes, 4K is 4 times larger than the standard HDTV in terms of resolution.

And also there is a larger one available, which is 8K. 

8K is 4 times larger than 4K, and 16 times larger than standard HDTV. 

Currently, 8K is only available in Japan. So, if you want to watch the 8K video you need to come to Japan. Recently, China and Korea are developing 8K imaging systems, so you may be able to see it in those countries, but Japan is the first country that has developed the 8K system.

The good thing about the super-ultra-high resolution is that you can display images on a very big monitor without degradation of the picture quality.

In fact, as you see on this slide, you may use 60-inch, 145-inch or even a theater size projection. For a certain purpose, the bigger, the better.

In my course, I will have a lecture about this 4K and 8K stuff later on.

//
//	Visualization basic - 9
//
(image:vizdata/viz-2/Slide9.PNG)

Well, this figure will appear in the 4K and 8K lecture, but as you see in this figure, you can compare how big the 8K is in comparison with SDTV or HDTV. 

SDTV is Standard Definition TV which is a spec of the old television in the past, almost 30 years ago. Now, as you know that HDTV is standard which has a resolution of 1920 by 1080.

So, when you use displays for your presentation, first you need to choose which resolution you will use. This slide only shows the resolution family in the video industry.

You may use other resolutions at your will, especially when you use a computer. In the computer industry, the resolution is very flexible, and many choices are available.

//
//	Visualization basic - 10
//
(image:vizdata/viz-2/Slide10.PNG)

If you use multiple displays, the resolution is theoretically limitless.

Here′re some examples. The left one is using a curved screen which dimension is horizontally 22 meters and vertically 6 meters, so it′s huge. Three 4K projectors are used to build this.

//
//	Visualization basic - 11
//
(image:vizdata/viz-2/Slide11.PNG)

Here′s another example of a multi-display system. This is used at the Cleveland Museum in the USA. 

It uses 23 HDTV tiles with a touch panel, and the dimension is horizontally 12 meters and vertically 1.5 meters. A number of people can use this big display with the touch panel at the same time.

(pause:2)

OK, then let′s watch a demo video of the Cleveland Museum.

//
//	Visualization basic - 12
//
(movie169:vizdata/viz-2/ClevelandGalleryOne.mp4)
(look at screen)

(pause:133)

//
//	Visualization basic - 13
//
(look at camera)
(image:vizdata/viz-2/Slide11.PNG)

According to my impression, museums in the USA are advanced in terms of the digital media like you have seen in the demo video. This Cleveland Museum employs a typical style of the exhibition with a mixture of the real exhibition and the virtual media associated with the real artifacts.


//
//	Visualization basic - 14
//
(image:vizdata/viz-2/Slide12.PNG)

The next thing is about the so-called immersive display. You know it very well because this is one of the trend technologies, that is VR, Virtual Reality.

Nowadays, there are many cheap but high-tech VR gears available, such as Oculus Rift, HTC Vive, PlayStation VR and so on. 

This head-mounted display, HMD is relatively old technology. I think it′s invented in the sixties and getting popular in the nineties and widely used after 2010.

Look at the left picture. This is a classic form of HMD, and you see the mask looks very heavy with a lot of wires and also comes with data gloves. It looks like an old science fiction movie but you can see the basic form of the gear doesn′t change a lot.

//
//	Visualization basic - 15
//
(image:vizdata/viz-2/Slide13.PNG)

Here′s another way of realizing immersive display. You don′t need to wear an HMD here. Instead, you are in a box room and each wall itself is a display equipped with a back projection. A front wall, two side walls, a floor, and a ceiling are all displays.

Also, you have a head tracking system, 3D mouse in your hands and multiple loudspeakers installed.

This equipment is called CAVE, which stands for Cave Automatic Virtual Environment, And it was fairly popular back in the nineties. Of course, it′s a huge system and super expensive, but many research organizations purchased this for their research at that time.

This is not for personal use but would be cool if you can design special games with it. 


//
//	Visualization basic - 16
//
(image:vizdata/viz-2/Slide14.PNG)

The next one is AR, Augmented Reality. 

This is also very popular nowadays and you should already know it very well. It′s a mixture of a real image and a real-time CG image using a camera and markers.

(pause:1)

Let′s watch a classic AR application.

//
//	Visualization basic - 17
//
(movie169:vizdata/viz-2/AR-Interactive-game-book.mp4)
(look at screen)

(pause:48)

//
//	Visualization basic - 18
//
(look at camera)
(image:vizdata/viz-2/Slide15.PNG)

Here′s the principle of AR. It uses a marker to recognize the target where the CG will be placed. It used to use some specific marker consists of simple lines, squares, and circles, however recently you can use a natural picture and even a 3D object as the marker.

Well, the camera is shooting this marker and the camera image is transferred to a computer to perform the camera parameter estimation. It calculates the position of the camera, x, y and z, and also the angles of the camera, rx, ry and rz. These six values are calculated by the computer in every video frame.

And these camera parameters are transferred to the CG rendering software to get the real-time rendered CG objects. And finally the generated CG objects are superimposed on the camera image, then it′s done.

(pause:1)

Nowadays, there′re many AR toolkits available so that you can build your own AR application quite easily. If it was a simple one, you don′t need to write any single code to make it.

In our course, we will have an AR workshop where you will make a simple AR application.


//
//	Visualization basic - 19
//
(image:vizdata/viz-2/Slide16.PNG)

The latest AR technology often uses a smartphone. The smartphone has a GPS system so that it can be used for the AR outdoor information service. 

Or you may use a kind of Google glass that you see at the right on the slide. As far as I know, they stopped producing the consumer version but releases the enterprise edition.


//
//	Visualization basic - 20
//
(image:vizdata/viz-2/Slide17.PNG)

The next topic is a panoramic image. The panoramic image is still used for the house viewing, for instance, in the real estate industry, and also in the sightseeing industry. And you know that the Google Street View uses panoramic images.

The use of a panoramic image is also classic. As far as I remember, the QuickTime VR was released around 1992, about 30 years ago. The QuickTime VR is a Web-based service using a panoramic image to enable a user to see a 360-degree view freely.

And in the panoramic image, as you can see on the slide, a straight line is transformed to a curved line and the curved line in the panoramic image is transformed back to the straight line in the output user view.


//
//	Visualization basic - 21
//
(image:vizdata/viz-2/Slide18.PNG)

This shows how the panoramic image is made. There′s a camera at the center shooting outside and there′s a cylindrical plane around the camera. And the camera image is mapped on the plane then you cut the cylindrical plane vertically and spread it out and you get one big horizontally-long plane. That is a panoramic image that covers a 360-degree view.

So, if you use this method, you cannot get the top image and the bottom image. 

(pause:1)

However, recently, owing to the advanced technology, the 360-degree image doesn′t have any blind area. You have the top and the bottom image also, it′s pure 360-degree view. This is done by stitching many images by many cameras shooting any direction including the top and the bottom.


//
//	Visualization basic - 22
//
(image:vizdata/viz-2/Slide19.PNG)

Nowadays not only a static panoramic image but also a panoramic movie is available. You can watch this 360-degree movie on YouTube. Try access YouTube and give "360-degree videos" in a search window then you will get tons of 360-degree movies.

This panoramic movie is filmed by a special camera. You can see the input devices on the slide, for example. Those are Theta by Ricoh and Allie by ICRealtek. 

Each device has two cameras attached together at its back. And each camera has a fish-eye lens to cover a semi-sphere of the outside view. And those two semi-spheres are stitched digitally to obtain a large 360-degree movie in real-time.

Those devices are not expensive, around 300 US$ or something.

However, the picture quality is still not so good. The built-in digital camera is mostly equivalent to a consumer digital camera. Since one camera covers a semi-sphere of the view which is a large area, the output image has far less resolution because it′s just a part of the semi-sphere.

Other than the resolution, it is pretty handy and a lot of fun.


//
//	Visualization basic - 23
//
(image:vizdata/viz-2/Slide20.PNG)

For the panorama viewing taken by the 360-degree camera, you may use an HMD instead of watching on a flat display. It would give you a more immersive nice feeling of the filmed environment.

You may use a Cardboard by Google. It′s a paper-made HMD with your smartphone, costing just only 10$.


//
//	Visualization basic - 24
//
(image:vizdata/viz-2/Slide21.PNG)

OK. Let′s move to another big topic, which is the Virtual Museum.



//
//	Visualization basic - 25
//
(image:vizdata/viz-2/Slide22.PNG)

Roughly to say, there are two kinds of virtual museums in terms of representing style. One is a real space virtual museum and the other is a Web-based one.

I have already shown an example of the real space virtual museum at the Cleveland Museum in the USA. 

It uses many technologies for virtual viewing such as immersive VR, big display, AR and so on. Basically, the real artifacts are there in the physical space and usually, the digital gears help the visitors appreciate the real artifacts in various ways.

(pause:1)

On the other hand, the Web-based virtual museum is open on the Internet. Recently, there is almost no real museum without having their own home page. The museum homepage itself has more or less virtual museum functions showing their collection on the pages. 

Not only showing images on the page, but other methods such as floor map navigation, panoramic walkthrough, artwork viewing, and 3D CG walkthrough could be also adopted. So, there are many levels of function provided on the Web site.


//
//	Visualization basic - 26
//
(image:vizdata/viz-2/Slide23.PNG)

Here′s the first example of the Web-based virtual viewing which is provided by the Louvre Museum in Paris. It uses a horizontal 360-degree panoramic image, a floor plan navigation, and information displays.

It is called Louvre Virtual Tours, and still available by accessing the link on the slide.

(pause:1)

OK, let′s watch it a little.


//
//	Visualization basic - 27
//
(movie:vizdata/viz-2/LouvreVirtualTour.mp4)
(look at screen)
(pause:92)


//
//	Visualization basic - 28
//
(look at camera)
(image:vizdata/viz-2/Slide24.PNG)

Compared with Louvre, you can find many excellent virtual museum services on the Internet in the USA. American museums are really advanced by all means. 

Here′s one of the examples which is the Smithsonian Museum in the US. It also uses a panoramic view with floor map and information display, but it′s far better than the one in the Louvre.

(pause:1)

OK, let′s watch it.


//
//	Visualization basic - 29
//
(movie:vizdata/viz-2/Smithsonian.mp4)
(look at screen)
(pause:105)



//
//	Visualization basic - 30
//
(look at camera)
(image:vizdata/viz-2/Slide25.PNG)

One of the greatest virtual museum services in the world is maybe this Google Arts and Culture run by Google. It used to be done as the name Google Art Project.

Technically, it uses the same panoramic image view, viewpoint transition but it is well organized. Maybe because it′s Google? Actually, the same technology in Google Street View is used in this museum project. The car equipped with multiple cameras used for the street view is coming into the museum space to film it.

(pause:1)

OK, let′s watch it.

//
//	Visualization basic - 31
//
(movie:vizdata/viz-2/GoogleArtCulture.mp4)
(look at screen)
(pause:68)
//
//	Visualization basic - 32
//
(movie:vizdata/viz-2/GigaPixel.mp4)
(look at screen)
(pause:170)


//
//	Visualization basic - 33
//
(look at camera)
(image:vizdata/viz-2/Slide26.PNG)

One of the very good things about Google is that first, they have a huge amount of money and power. This Google Arts and Culture becomes possible because Google has a lot of connections with many museums all over the world. 

Since Google is a super famous big IT company, it might be easy to establish Give and Take between Google and museums. The museums let them come to the museum and film the museum and even get Gigapixel data of their collection, and instead, the museum can reach a massive amount of people on the internet for advertisement. 


//
//	Visualization basic - 34
//
(image:vizdata/viz-2/Slide27.PNG)

Well, until now, almost all the virtual museum services are done by the 360-degree static image of the museum space and the 2D picture data with high-resolution. On the other hand, there are some services using 3D real-time CG for the virtual museum, but it′s not many. 

This is quite a straightforward way. The entire museum space is purely virtual and is modeled with 3D CG. And the artworks such as painting, photographs, and sculptures are displayed in the CG space so that a user can walk through freely to appreciate the artworks. In fact, this is exactly what you will do in this course.

(pause:1)

There are several reasons why this method is not so popular. One of the factors might be computer power to render real-time CG. Sometimes it requires high-speed GPU if the CG model is very fine with high-resolution texture.
 
The other reason might be usability. Ordinally people may not be familiar with navigating themselves in a 3D CG space although gamers feel pretty much comfortable with First Person Shooting.

(pause:1)

However, some of the services are available on the internet. Here′s the example. A German company GRAPHTWERK provides 3D CG walkthrough space on a user′s Web browser. They use Unity Game Engine to construct the service, but some of the applications on the site become old and not available. 

Try access to the site and see what they′re doing.


//
//	Visualization basic - 35
//
(image:vizdata/viz-2/Slide28.PNG)

Here′s another example of showing 3D objects using 3D CG. This is the Web site of Hampson Museum in the US. They scan their 3D object collection and open it to the public via the Web site. 

(pause:1)

Let′s watch one example.

//
//	Visualization basic - 36
//
(movie:vizdata/viz-2/Hampson3DObject.mp4)
(look at screen)
(pause:100)

//
//	Visualization basic - 37
//
(look at camera)
(image:vizdata/viz-2/Slide29.PNG)

This is not a virtual museum but it′s an interesting way to use 3D real-time CG for medical use. It′s the Virtual Autopsy developed by Linköping University in Sweden.

A human dead body or it could be a mummy is scanned by a CT scanner or MRI scanner. The obtained voxel data can be displayed in many ways on a high-resolution touchscreen installed on an autopsy table. You can observe inside of the body freely.

(pause:1)

Let′s watch the demo video.

//
//	Visualization basic - 38
//
(movie:vizdata/viz-2/The-Virtual-Autopsy-Table.mp4)
(look at screen)
(pause:133)

//
//	Visualization basic - 39
//
(look at camera)
(image:vizdata/viz-2/Slide30.PNG)

Finally, I will introduce you just only one topic which I think it′s interesting in terms of the User environment on the internet.

//
//	Visualization basic - 40
//
(image:vizdata/viz-2/Slide32.PNG)

This is a project using collective knowledge on the Internet. The painting on this slide is an oil canvas created by a Dutch painter Johannes Vermeer who is a very famous classic painter in the seventeenth century.

And the experiment is done by making a Web site showing this kind of world-famous paintings to the public where the visitors are able to give free keywords associated with the painting through a web form to submit.

Then, after several months, those keywords are collected and ranked, then each painting gets five keywords made by anonymous visitors.

Here, you can see five keywords for the Vermeer′s famous painting. Those are

Tranquil, interior, water jug, woman and shadow.

I think this is a very interesting result. Because, since this painting is super famous, and it′s in the collection of the super famous museum where a few professional curators are working, if the curators made five keywords to this painting, those would not be like this. 

They must figure out other keywords, something more professional. Those should have not had, for instance, Tranquil as a keyword.

This means that there might be a potential to make something new, at least to find any different perspective by opening those established artworks to the public.

This attempt is not very new but I think things are going toward this direction in the twenty-first century.

//
//	Visualization basic - 41
//
(image:vizdata/viz-2/Slide33.PNG)

All right, OK, that′s all for now!

See you next time!



(ending start)
Well, do remember my name until next time.
Adios
(ending end)






