FlatCam prototype eliminates lens using pinhole-like sensor mask

Thanks.

I was referring to your comment "that 50×50 pixels may be — theoretically — achievable" with a 512x512 sensor. I naively thought that limit would scale. In other words, with a 5,000,000 x 5,000,000 sensor AND with a correspondingly finer detailed mask, the theoretical limit may be 50,000 pixels x 50,000 pixels. Which is pathetic.

I now see that diffraction would get in the way of even achieving that resolution.

Well, I did not try to redesign their system myself — so I’m not 100% sure about the effects of diffraction in their design.

In this sensor, the image at every sensel is formed by a COMBINATION of the individual “squares” of the mask. From this, it is clear that the resolution cannot be better then the blur created by every particular square. However, the contribution of diffraction may be somewhere “in between” the contribution of an individual square, and the contribution of the full mask. (This may also depend on the used algorithm of de-masking.)

So I would not be surprised that the contribution of diffraction may be 2 (or 3? probably not stronger than this) times smaller than what an “individual masklet” would create.

But I already took this into account in my estimate of 50×50 resolution! (And, in their particular design, the diffraction is way below the defocus-blur, so these fine points do not matter.)

Thanks. Diffraction won't be an issue at these resolutions but seems like it would if you try to significantly scale up resolution.

Diffraction by each pinhole is NOT an issue, as there is only one sensor behind each pinhole, and it does not matter where on each tiny sensor a photon hits.
The resolution lies in the DISTANCE between each pinhole AND how accurately the cones of recorded light is known for each pinhole (the exact shape and angle of each cone) - optimally each hole had exactly known size and shape, and exactly known distance to the sensor - this is (of course) not physically feasible, and thus add errors to the calculations

Diffraction by each pinhole is NOT an issue, as there is only one sensor behind each pinhole, and it does not matter where on each tiny sensor a photon hits.
The resolution lies in the DISTANCE between each pinhole AND how accurately the cones of recorded light is known for each pinhole (the exact shape and angle of each cone) - optimally each hole had exactly known size and shape, and exactly known distance to the sensor - this is (of course) not physically feasible, and thus add errors to the calculations

Have a look at this sketch on the principle on what is getting recorded in each sensor (one dimension only)
http://www.dpreview.com/galleries/4515373657/photos/3398315/flatsensor_principle

Thank you for the diagram, it is very helpful. If we were to scale the resolution of the sensor and the number of pinholes on the mask from 512x512 to 5000 x 5000, why wouldn't diffraction be an issue. At these smaller sizes, wouldn't photons start hitting other pixels?

@Alex Permit
The point is that it does not matter WHERE behind each pinhole a photon hits, as long as it is detected. So diffraction is NOT the problem here - it is far from problem free though.

The more pinhole-sensors the more complex the calculations, and thus the higher accuracy is needed in the numerical representation to calculate what the motive must have looked like.
(So it will also be very sensible to noise in the recorded signal)
So the narrower the cones are the better, as this means less overlap.
In theory the perfect version talking resolution and math would be a sensor as big as the motive and cones collapsed to cylinders, That would ONLY record light exactly perpendicular to the sensor.

As I see it the worst problem with the method is that the sensor needs to be quite large to handle objects in the distance. The closer the motive is to the system the fewer pinholes will record each element of the motive, thus easing the calculus.

@EskeRahn:

Diffraction ALWAYS matters (with small apertures). It does not matter what you put BEHIND the pinholes; it is enough that diffraction scrambles the rays of the incoming lightfield.

With a 30µ pinhole, you CANNOT get resolution above about 1° (with visible light).

About your illustration: can you please mark the page/column/paragraph of the Arxiv paper it would match? I see no relation of your picture to what the paper discusses.

(BTW, I did not look VERY careful into the paper. It looks very similar to the “bright 3D display” idea mentioned here some time ago — and THAT paper I did read through [it was fun!]. If so, your picture has NO RELATION to the idea of the device.)

@ilza
Diffraction 'only' matters if it matters where the light ends after the aperture and it does not with this system. There is only one 'common' sensor for each pin hole, and it does not matter if it hits this tiny sensor as a fuzzy ring or a pin sharp point

But of course this does not mean we can get arbitrarily fine resolution, as the sensors and pin holes can not be arbitrarily small.

On the article vs my sketch, look at the right part of Fig.2 top p3. (my sketch is simplified with identical holes)

@EskeRahn, what if the sensor pixels and pinholes are so small that the light starts hitting adjacent pixels? I imagine thats not an issue with a 512x512 sensor, but perhaps it is for a 5000x5000 sensor?

@Alex Permit
Correct, as I said:
"But of course this does not mean we can get arbitrarily fine resolution, as the sensors and pin holes can not be arbitrarily small."

I do not know where the limit is, but it is not with the number of pin holes as such, but the grid-size of the pin holes.
(But of course if we assume the same overall size, then too many pin holes, would also mean that they got too close)

@EskeRahn:

I’m starting to think that you are missing a lot. First of all, diffraction does not care what is behind the entry pupil: when the incoming rays are scrambled, there is no way to unscramble them. Second, the smaller the pinholes, the WORSE is the resolution (diffraction-wise).

For example, with 1µ pinholes, and 60° field of view, there is non way to get the resolution more than 4×4 pixels (maybe even 2×2 — I’m always forgetting what the coefficient is!).

Actually absolute IQ matters very little for me and you. We can figure out most anything with just a little vision. But for machines, IQ really is a big deal. They are not nearly smart enough yet to do without it in any critical operation.

If a pinhole camera is a camera then a multiple pinhole device for capturing images would be a camera. That sounds a lot like photography to me.

I do like a good debate about semantics.

Photography being an art vs. image recognition being engineering.

Fair enough, but I don't think there is a debate here. Photography has a pretty broad definition no matter where you look up the word. This is photography. I think you were trying to say that this isn't yet viable for mainstream photographers in their workflow, but saying it isn't photography isn't pragmatically or semantically correct.

It will be used as a machine sensor; It will stream data to computational processes. It will not "record light on to a medium".

yes, but the paper seems to miss to mention on what platform they got that processing time (supposed on a reasonable fast desktop PC) ... anyway, it's relevant only about order of magnitude of the value, not the exact value in itself...

Facial recognition?

Leonp, you're the only one who *got* my point. Congrats. There's still hop+e in this world after all.