The world's First 12K. The True, Resolution.
The world's first "true 12K" real-time 360° camera
that can provide more realistic and smooth VR images
1with delicate and high-quality images.
At this stage, VR is not yet well commercialized. The major problem is in image quality. When you look at a low quality video image, your brain compensates for the difference from the actual screen you are looking at. People have the ability to correct the image they are looking at to the image they want to see. It is very tiring because of the extra use of the brain. The "VR sickness." This brain conversion is a major cause of most cases of "VR sickness.
Famously, older computers described the text with a great deal of omission. If you look closely, it is completely different, but your brain compensates. This is why reading a book on a computer screen or tablet is so tiring, unlike looking at a book.
The most famous brain correction is the typoglycemia. It is possible to read letters that are written completely differently. This is because the brain corrects the letters to the ones you want to read.
Currently, 4K cameras are the most common way to shoot VR, but when viewed in VR, the image is not the high quality image seen on a 4K TV. While 4K TVs have clear outlines, VR requires 360-degree image data, including parts that are not visible, so it is necessary to think differently than with ordinary flat displays.
source:https://xr-creek.jp/topics/vr4kresolution/
In the case of a 4K 360-degree image, 4K means that the data includes the part of the image that is not visible at that moment, so the image actually visible on the VR goggles is less than 4K. Furthermore, to create a 360-degree VR, it is created using a fisheye lens.
The 8K VR cameras that are readily available today utilize two 4K sensors, i.e., two cameras to create VR, but use a fisheye lens to create a 360-degree image, rounded at the maximum vertical value. As a result, only 2K equivalent pixels are actually used. The rest is discarded.
4The real resolution of 4K is 3,840X2,160 = 8,294,400, but in the case of VR, the diameter is 2,160, so 1,080X1,080X3.14 = 3,662,496, which is actually only 2.5K level resolution. 8K VR cameras can only achieve the real 2.5K equivalent In fact, an 8K VR camera can only achieve the equivalent of 2.5K image quality in real terms!
As of June 2022, two manufacturers have 12K cameras available, but they announce of 12K by combining 8 unit of 4K sensors.
Doing the same math, for VR, the diameter is 2160, so
1080X1080X3.14 = 3,662,496 (one sensor) 3,662,496 X 8 = 29,299,968.
The Oculus Quest2, a representative of VR goggles, can view images equivalent to 4K in one eye and 8K in both eyes.
Source:https://xr-creek.jp/topics/vr4kresolution/
In order to view this one plane in 8K, theoretically, 8K x 6 planes of video data would be required for the entire image. However, when the cube is reconstructed as a sphere, the corners will overlap in reality, so the guideline is 8K x 4 planes = 16K (*ideal value).
However, many VR goggles come standard with the ability to detail only what you see with your eyes and reduce the quality of that part of the image, so that in effect 12K is sufficient to achieve the desired quality. You will understand that current 8K and 12K cameras do not have enough resolution at all.
The first major problem is that there are no 12K cameras for VR. The actual method is to combine eight 4K sensors to create a pseudo-show, but as mentioned above, the actual maximum resolution is 8K.
Another major reason is that there is no chip that can do the stitching (Multiple images are combined and processed into a single image.)for images with resolutions beyond 8K.。
The next problem is the physical problem of transmitting the video. This is the biggest problem.
To transmit 12K 60 frames per second video 12K60P without compression requires a 144 Gbit transfer rate.
1G at the maximum for home Internet connections and the 5G networks that have recently emerged, so 144 lines of transfer speeds are required, which is not realistic!
Therefore, compression is necessary. Using current mainstream compression technology, it is possible to compress down to one-sixth. The number of frames in a video can be compressed down to 4.8G by reducing the frame rate to 12P instead of 60P. But even this is not realistic!
HeartCore's [ truRes-12K ] has developed and released a camera, chipset, stitching processing technology, and compression technology simultaneously.
[Sample System Features]
• Live images from two cameras with 8K resolution can be combined on a playback PC to display 360° images with up to 16K resolution.
• Compresses images from two cameras and transfers them to a server. Stitching is performed on the server, and the video is converted to images on the server for distribution, and reprocessed to a video of approximately 12p per second. The converted video is then played back on a playback PC.
• Playback Video and audio together on a PC.
• Can be viewed on two HMDs at the same time.
▼truRes-12K Camera System Overview(SAMPLE)
▼Camera Overview
• Two 8K images are acquired using two cameras; the two images are asynchronous.
• Power to the camera section is supplied by PoE.
• Audio is input from the microphone's LINE OUT terminal to the camera's MIC terminal.
• In this sample system, there are areas that cannot be imaged directly above/below, which do not exist in the actual commercial version, because the video is shot with a fisheye lens for still photography.
▼Sample Camera Specifications
7It utilizes a 7,680 X 7,680 square image sensor; for VR, 3,840 X 3,840 X 3.14 = 46,301,184 is one camera.
Because two are utilized, the total is X2, for a total of 92,602,368, which is effectively 10K worth of images (or 16K by normal industry count).
The sample presented here has half this resolution, but since there is no chipset to process stitching, it is transferred to a PC, stitched on the PC, and processed into a single movie. Due to performance limitations, the resolution will be 10 frames per second.
For transmission, the video compression equipment performs hardware compression to 1/20th of the original size. 12K12P video can be compressed down to 1.44G. This can be transferred over two 1G networks or four 5G lines.
10G Internet connections are becoming more common, and 10G can be handled even on a best-effort basis.
▼Comparison of other products during live streamings
With the release of this video compression technology and a true 12K VR camera, we will be able to realize a technology that is 10 years ahead of its time this year. The applications are considered for VR travel, VR commerce, medical care, factories, remote operations, and many others.
The use of VR is expected to advance further with the full-scale launch of "5G" services by mobile carriers. 5G, with its features of "high capacity, low latency, and multiple connections," is expected to enable smooth delivery of VR, which requires large amounts of data to be exchanged, and to reduce the time lag between live delivery and the local site.
With the arrival of the 5G era in full swing, content holders should be able to expect further developments that leverage the appeal of the content they own, with "immersive experience" as the keyword.
HeartCore aims to be a platformer in this field.
