以复眼为灵感开发的超薄阵列相机

~Xenos peckii复眼和节肢动物复眼的比较，以及受Xenos peckii眼启发的相机设计~

A comparison of the eye of Xenos peckii and the arthropod eye, and a camera designs inspired by the eye of Xenos peckii

大自然中的复眼因其在广阔视野中捕捉视觉信息的卓越能力广受研究人员关注。在这些复杂的光学系统中，Xenos peckii，一种扭纹虫昆虫，的复眼和节肢动物的复眼的高效的视觉感知便是一个引人注目的例子。本报告旨在探索和比较Xenos peckii的复眼和节肢动物复眼的独特特征，并深入探讨受这些自然奇观启发的相机系统的设计和开发。通过理解这些复眼的结构和功能方面，我们可以揭示相机技术领域的新见解和潜在应用。通过这次探索，我们将发现复眼如何激发创新的成像系统解决方案。

Compound eyes found in nature have fascinated researchers for their remarkable ability to capture visual information across a wide field of view. Among these intricate optical systems, the eye of Xenos peckii, a strepsipteran insect, and the compound eyes of arthropods stand out as captivating examples of efficient visual perception. This report aims to explore and compare the unique characteristics of Xenos peckii's compound eye and the arthropod eye, as well as delve into the design and development of a camera systems inspired by these natural wonders. By understanding the structural and functional aspects of these compound eyes, we can uncover novel insights and potential applications in the field of camera technology. Through this exploration, we will discover how compound eyes can inspire innovative solutions for imaging systems.

X. peckii的独特复眼 

X. peckii的复眼与典型的节肢动物复眼相比具有独特的特点。与果蝇等其他小昆虫的眼睛有700多个小结构不同，X. peckii的眼睛大约有50个较大的透镜（1）。X. peckii中的每个透镜直径约为65 µm，覆盖了相当于果蝇中15个透镜的面积（图1，Buschbeck et al., 1999）。

The compound eye of X. peckii, a strepsipteran insect, exhibits unique characteristics compared to typical arthropod eyes. Unlike other small insects like Drosophila melanogaster, which have over 700 facets in their eyes, X. peckii has around 50 larger lenses  (1). Each lens in X. peckii is approximately 65 µm in diameter and covers an area equivalent to 15 lenses in D. melanogaster (Figure 1, Buschbeck tet al., 1999).

X. peckii的眼由称为眼小区的个体单位组成，每个眼小区都包含一个自己的视网膜，被色素囊围绕（图1，Maksimovic等人，2007）。光学测量结果表明，这些眼小区独立地处理投射到它们各自视网膜上的视觉信息。与节肢动物眼不同，其中每个小结构都有8至10个光感受器，X. peckii的扩展视网膜含有100多个感受器细胞（2, 4）。因此，X. peckii眼中的视野被划分为“块”，而不是被分解为单个点。这种独特的结构使得每个眼小区能够将视野中的物体清晰地聚焦（图3.A和B，Buschbeck等人，1999）。

The eye of X. peckii consists of individual units called eyelets, each containing its own retina surrounded by a pigmented cup (Figure 1, Maksimovic et al., 2007). Optical measurements suggest that these eyelets function independently to process the visual information projected onto their respective retinas. Unlike arthropod eyes, where each facet contributes to a single sample point with 8 to 10 photoreceptors, X. peckii's extended retina contains over 100 receptor cells (2, 4). Consequently, the visual field in X. peckii's eye is divided into "chunks" rather than being decomposed into individual points. This unique structure enables each eyelet to bring an object in the visual field into clear focus (Figure 3. A and B, Buschbeck tet al., 1999)

X. peckii眼中眼小区的组织方式导致了复眼下方视觉系统的神经解剖排列的差异。来自每个眼小区感受器细胞的投射形成一个在板状区域（图2A）终止并绕其轴线扭曲（图2B，Buschbeck等人，1999），从而使每个视网膜在板状区域上的空间表示大约旋转了180度。

The organization of eyelets in X. peckii leads to differences in the neuroanatomical arrangement of the visual system beneath the compound eyes. The projections of the receptor cells from each eyelet form a nerve that terminates in the lamina (Figure 2A) and twists around its axis (Figure 2B, Buschbeck tet al., 1999), resulting in a rotation of the spatial representation of each retina on the lamina by approximately 180 degrees.

基于这些解剖数据，研究人员提出了X. peckii眼中视觉处理的模型（图4B），说明它与传统复合眼（图4A）的显著差异。使用颜色来表示图像的表示。在左侧的并置眼中，每个光学单位只表示一个采样点。图像的相邻点在视网膜和板状区域的水平上以及旁边表示。在右侧显示的X. peckii眼中，图像通过多个眼小区查看，每个透镜捕捉部分图像。由于每个透镜都具有图像反转作用，整个图像在视网膜层次上失去了连贯性，但在视网膜和板状区域之间的神经扭曲中得以恢复（2）。这种复合透镜眼睛的优势在于具有高光聚集能力和图像分辨率的结合，否则在小昆虫中很难实现（1）。

On the basis of these anatomical data, researchers have proposed a model for visual processing in X. peckii's eye, (Figure 4B) illustrating its substantial deviation from conventional compound eyes (Figure 4A). Color is used to depict the representation of an image. In the apposition eye on the left, each optic unit only represents one sample point. Neighboring points of an image are represented next to each other at the level of the retina as well as the lamina. In the eye of X. peckii shown on the right, the image is viewed through multiple eyelets, with each lens capturing a partial image. Because each lens is image-reversing, the coherence of the entire image is lost a the level of the retina, but is restored by the physical twisting of the nerves between the retina and the lamina (2). The advantage of such a composite-lens eye lies in a combined high light-gathering ability and image resolution that otherwise would be difficult to achieve in small insects (1).

基于X. peckii眼的高对比度和高分辨率成像的超薄阵列相机

Ultra-thin arrayed camera for high-contrast and high-resolution imaging inspired by the eye of X. peckii

在最近的研究中，人们采用各种微纳制造方法结合受生物启发的相机，创建了微透镜阵列。这些方法，如热流变、喷墨打印和三维直接激光写入，仍在开发中，以解决微透镜之间的光学串扰问题，以实现高对比度成像。传统的光吸收器，如叠层玻璃光阑阵列或加工的光折射器阵列，在减少光学串扰方面也有其局限性（5,6）。

In recent studies, various micro-fabrication methods have been employed in combination with biologically inspired cameras to create micro-lens arrays. These methods, such as thermal reflow, inkjet printing, and 3D direct laser writing, are still being developed to address the issue of optical crosstalk between micro-lenses in order to achieve high-contrast imaging. Traditional light absorbers, like glass stacked diaphragm arrays or machined baffle arrays, also have their own limitations in reducing optical crosstalk (5,6).

受X. peckii独特视觉系统的启发，韩国科学技术高等研究院的研究人员开发了一种超薄阵列相机。该相机由多层孔径阵列（MAAs）、倒置微透镜阵列（iMLAs）和平面CMOS图像传感器上的间隙间隔器组成。

Taking inspiration from the unique visual system of X. peckii and aiming to overcome these technical limitations, researchers from the Korea Advanced Institute of Science and Technology have developed an ultra-thin arrayed camera. This camera consists of multilayered aperture arrays (MAAs), inverted micro-lens arrays (iMLAs), and gap spacers on a planar CMOS image sensor.

MAAs由堆叠的黑色聚合物圆形图案组成，用作圆柱形针孔阵列。这些阵列在可见光谱范围内高效吸收光线，显著减少了微透镜之间的光学串扰。iMLAs位于MAAs中的针孔下方，与传统向上微透镜阵列相比，提供了相对较大的视场（FOV），因为它们允许来自前玻璃窗口的折射光以额外的角度进入单个微透镜。MAAs的厚度为60微米，孔径直径为35微米，FOV约为70°（3）。

The MAAs are composed of stacked black polymer circular patterns that serve as cylindrical pinhole arrays. These arrays efficiently absorb light across the visible spectrum, significantly reducing optical crosstalk between micro-lenses. The iMLAs, placed beneath the pinholes in the MAAs, offer a relatively larger field of view (FOV) compared to traditional upward micro-lens arrays because they allow refracted light from a front glass window to enter individual micro-lenses at an additional angle. With a thickness of 60 µm and an aperture diameter of 35 µm, the FOV of the MAAs is approximately 70° (3). 

通过iMLAs和MAAs，远场平面上的物体被成像到每个通道上。由于每个通道的视差很小，这些阵列图像是均匀的，但稍有不同。最后，可以使用右侧显示的超分辨率算法从阵列图像中重建出单个高分辨率图像，该算法通过最小化图像与数据之间的LP范数（p=2），并结合基于先验知识的正则化项和λ权重。

Through the iMLAs and MAAs, objects located in the far-field plane are imaged on each channel. Due to the small visual disparity of each channel, these array images are uniform but are slightly different. Lastly, a single high-resolution image can be reconstructed from the array images by using a super-resolution algorithm shown on the right that  minimizes the LP norm (p=2) between the images and the data, incorporating a regularization term based on the prior and a lambda weight.

实验结果表明，相机的多层针孔结构通过消除光学串扰实现了高对比度成像。通过仅通过MAAs、仅通过iMLAs和组合的MAAs和iMLAs（称为MOE）捕捉532纳米激光束的分割图像，显示MOE产生了不带有任何微透镜之间的光学串扰的锐利聚焦光束（图3a）。MOE的强度分布曲线（图3b）还表明锐利的峰值信号，没有噪声或串扰，明确显示了光束聚焦和光线阻断。此外，通过仅使用iMLAs和MOE捕捉的棋盘格图像（图3d）清楚地表明MOE提供了比仅使用iMLAs更高的对比度。

Experimental results demonstrate that the multilayered pinhole structure of the camera enables high-contrast imaging by eliminating optical crosstalk. Sectioned images of a 532-nm laser beam passing through the MAAs only, iMLAs only, and the combined MAAs and iMLAs (referred to as MOE) show that the MOE produces sharp focused beams without any optical crosstalk between micro-lenses (Figure 3a). Intensity profiles of the MOE  (Figure 3b) also indicate a sharp peak signal without noise or crosstalk, clearly demonstrating beam focusing and light blocking. In addition, the images of a checkerboard captured through the iMLAs only and with the MOE (Figure 3d) clearly indicate that the MOE provides higher contrast than only the iMLAs.

通过捕捉和重建骰子的图像，演示了超薄阵列相机的重建能力（图4a）。重建的图像显示边缘清晰度和对比度增加，调制传递函数（MTF）随合并通道图像数量呈对数增加。研究人员还通过使用欧几里德距离测量目标对象（图4d）和获取的图像（图4e和f）之间的颜色差异，对超薄阵列相机的重建图像进行了比较，显示出单通道图像的归一化颜色差异为0.31，重建图像为0.08。

The camera's reconstruction capabilities are demonstrated through capturing and reconstructing images of a dice (Figure 4a). The reconstructed images exhibit increased edge sharpness and contrast, and the modulation transfer function (MTF) logarithmically increases with the number of merged channel images. The researchers also captured images of a red parrot to measure the color differences between the target object (Figure 4d) and acquired images (Figure 4e and f) using the Euclidean distance, which exhibits a normalized color difference of 0.31 for the single channel image and 0.08 for the reconstructed image.

将超薄阵列相机应用于光学显微镜和工业检测等领域可以为高分辨率和高对比度成像提供新的解决方案。此外，这种受X. peckii复合眼启发的相机设计还为未来的显微成像和摄影技术提供了新的思路和创新潜力。

The reconstructed images from the ultra-thin arrayed camera were further compared with those from a commercialized compact camera. Compared to commercial compact cameras or mobile cameras, the ultra-thin arrayed camera exhibited exceptional figures of merit, including image resolution, FOV that is 1.5 times of the FOV of a commercial camera, a substantial improvement of 5.41 times in the TTL and cost-effectiveness. Such properties of the ultra-thin camera provides new opportunities for diverse mobile, surveillance, or medical applications.

总结

X. peckii的复眼和节肢动物眼在结构和功能上存在差异。X. peckii的眼由较少数量的大型透镜组成，每个透镜覆盖了较大的视野，并具有更多的感受器细胞。这种结构使得X. peckii能够以高分辨率和高对比度聚焦视野中的物体。

基于X. peckii眼的启发，研究人员开发了一种超薄阵列相机，利用多层孔径阵列和倒置微透镜阵列来实现高对比度和高分辨率成像。实验证明，该相机具有消除光学串扰的能力，并能够重建出锐利、清晰的图像。

这些研究揭示了复合眼在相机技术中的潜在应用，为显微成像和摄影领域的创新提供了新的思路和解决方案。通过借鉴自然界的设计原理，我们可以进一步改进和发展相机技术，以实现更高质量的图像和更广阔的应用领域。