Covering from photography to depth and spectral estimation, diverse computational imaging (CI) applications benefit from the versatile modulation of coded apertures (CAs). The lightwave fields as space, time, or spectral can be modulated to obtain projected encoded information at the sensor that is then decoded by efficient methods, such as the modern deep learning decoders. Although the CA can be fabricated to produce an analog modulation, a binary CA is preferred since more straightforward calibration, higher speed, and lower storage are achieved. As the performance of the decoder mainly depends on the structure of the CA, several works optimize the CA ensembles by customizing regularizers for a particular application without considering the critical physical constraints of the CAs. This work presents an end-to-end (E2E) deep learning-based optimization of CAs for CI tasks. The CA design method aims …