[PATCH v3 0/3] media: Add support for the Sony IMX681

Sergey Lebedev posted 3 patches 2 weeks, 1 day ago
.../bindings/media/i2c/sony,imx681.yaml       | 107 +++
MAINTAINERS                                   |   8 +
drivers/media/i2c/Kconfig                     |  10 +
drivers/media/i2c/Makefile                    |   1 +
drivers/media/i2c/imx681.c                    | 884 ++++++++++++++++++
drivers/media/pci/intel/ipu-bridge.c          |   2 +
6 files changed, 1012 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/i2c/sony,imx681.yaml
create mode 100644 drivers/media/i2c/imx681.c
[PATCH v3 0/3] media: Add support for the Sony IMX681
Posted by Sergey Lebedev 2 weeks, 1 day ago
The Sony IMX681 is the user-facing camera on the Microsoft Surface Pro 11 for
Business (Intel Lunar Lake, IPU7), enumerated as ACPI device SONY0681. Without
a driver the camera does not appear at all - not as a degraded device, not at
all.

  1/3  dt-bindings: media: Add Sony IMX681            (mine)
  2/3  media: i2c: Add Sony IMX681 sensor driver      (Andre Gilerson's)
  3/3  media: ipu-bridge: Add Sony IMX681             (mine)

The driver is Andre's work, reverse-engineered from I2C traces taken under
Windows. I am carrying the submission, not the code: his Signed-off-by is on
2/3 with mine beneath it as the person passing it on. He is away until 28
September, so replies to review in the next few weeks will come from me. I have
the hardware, the instrumentation is scripted, and I would rather measure what
you ask for than argue about it.

Changes in v3
=============

One fix, and a retraction that matters more than the fix.

The automated review of v2 caught a regression I introduced between v1 and v2,
and it is right. v1 tested pm_runtime_get_if_active() with "!", which I called
a bug - it reads -EINVAL as success and then puts a reference it never took.
v2 changed it to "<= 0", which fixes that and breaks something v1 had right:
without CONFIG_PM the helper returns -EINVAL unconditionally,

  static inline int pm_runtime_get_if_active(struct device *dev)
  {
          return -EINVAL;
  }

so "<= 0" drops every control write - exposure, gain, blanking, test pattern -
silently. With no runtime PM the sensor is powered from probe and never
suspended, so those writes still have to go out.

v3 uses the shape ov64a40 already has, which is the only fully correct caller
of this helper in drivers/media/i2c:

        pm_status = pm_runtime_get_if_active(imx681->dev);
        if (!pm_status)
                return 0;
        ...
        if (pm_status > 0)
                pm_runtime_put(imx681->dev);

0 skips, 1 applies and puts, -EINVAL applies and does not put. Ten lines
changed, six of them comment. 1/3 and 3/3 are byte-identical to v2.

Withdrawing an offer made in the v2 cover letter. It said the "!" form is in
gc05a2, gc08a3, imx283, lm3560, og0ve1b and ov6211 and offered a cleanup
converting them. Please do not take that up: converting them to "<= 0" would
break all six exactly as it broke this driver. If a cleanup is still wanted it
should take ov64a40's shape, and I will send one only if a maintainer asks.

The reasoning behind that offer was wrong in a way worth naming, because it is
easy to repeat. I compared against i915, xe, ipa, ivpu and arm-smmu, which do
test "> 0" or "<= 0" - but they sit where CONFIG_PM is effectively mandatory,
so the third return never reaches them. drivers/media/i2c has no such
guarantee. I also read ov64a40 at the time, wrote it down as a variant of the
same thing, and did not look at its second half.

Blast radius of the v2 defect, since it is fair to ask: nil in practice, and
not to my credit. v2 was on the list for four hours and applied nowhere, and on
x86 with suspend PM_SLEEP is def_bool y and selects PM, so every machine this
sensor exists on forces CONFIG_PM on and could not have hit it. The real
exposure was the cleanup offer, which pointed at six drivers that ship. That is
why it is retracted at the top of this list rather than the bottom.

My full answer to the v2 review, which also carries a report against
ipu-bridge that the same run raised and that is not this series' to fix:

  https://patch.msgid.link/20260909203527.90372-1-lsa.uz@pm.me

Link to v2: https://patch.msgid.link/20260909193953.86192-1-lsa.uz@pm.me
Link to v1: https://patch.msgid.link/20260909174240.80023-1-lsa.uz@pm.me

The four fixes v2 introduced, carried into v3 unchanged
=======================================================

1. devm_regulator_bulk_get() failure is now fatal, via dev_err_probe().

   v1 logged a dev_dbg and continued, and imx681_power_on() then called
   regulator_bulk_enable() on an array nothing had filled in. That is worse
   than it sounds: _regulator_bulk_get() unwinds with regulator_put() and
   leaves the pointers where they are, so a get that fails partway puts freed
   regulators into supplies[] rather than NULLs.

   This reverses a deliberate choice - the code carried the author's comment
   "Continue without regulators - INT3472 may handle power" - so the reasoning
   is worth stating. The get does not fail for supplies the firmware leaves
   undescribed; it hands out dummies, which is what this machine shows for
   dvdd and dovdd. A real failure is therefore exceptional, and the likely one
   is -EPROBE_DEFER, which is what dev_err_probe() exists to handle. Andre can
   overrule me when he is back on the 28th.

2. pm_runtime_get_if_active() is tested against all three of its return values.

   v1 wrote "if (!pm_runtime_get_if_active(dev))", which reads -EINVAL as
   success, touches the sensor over I2C and then pm_runtime_put()s a reference
   it never took. The kerneldoc is explicit that on -EINVAL "the usage_count
   will remain unmodified".

   Beyond this driver, and offered rather than assumed: the same "!" form is in
   gc05a2, gc08a3, imx283, lm3560, og0ve1b and ov6211. Everywhere else in the
   tree that I looked - i915, xe, ipa, ivpu, arm-smmu - it is tested as "> 0"
   or "<= 0". If a cleanup for drivers/media/i2c is wanted I will send one
   separately; it is not in this series.

3. pm_runtime_put_noidle() added on the probe error path, which otherwise
   called pm_runtime_set_suspended() while still holding the reference taken by
   pm_runtime_get_noresume().

4. V4L2_CID_ANALOGUE_GAIN no longer advertises or applies digital gain.

   v1 advertised 0..1020 and made up everything above code 960 by writing
   IMX681_REG_DIGITAL_GAIN - the register V4L2_CID_DIGITAL_GAIN also writes.
   The clash is not limited to that top end, though, and that is what made it
   easy to miss: the analogue handler wrote the digital register on *every*
   call, putting back the 1.0x it had initialised the local to whenever the
   requested code was at or below 960. So setting digital gain to 4x and then
   moving analogue gain anywhere at all silently reset it, with no error and
   with the DIGITAL_GAIN control still reporting 4x.

   Clustering the two controls would have stopped them overwriting each other.
   I did not do that, because it would then need an invented rule for what a
   user means by analogue 1020 together with digital 2x, and there is no honest
   answer: they are two names for one register. Removing the overlap answers the
   clash and the range question together, which is why they turned out to be one
   question rather than two.

   So ANALOGUE_GAIN is now 0..960 - 1x to 16x, what the analogue stage actually
   does - and DIGITAL_GAIN owns 0x020E alone. The driver's own define admitted
   the problem: IMX681_ANA_GAIN_MAX carried the comment "combined
   analog+digital", and V4L2 has no control for combined gain. 256x is still
   reachable, as 16x analogue times 16x digital, which is what the hardware was
   doing all along; the difference is that userspace can now see which half is
   which, and an AE loop that prefers analogue gain in order not to amplify
   noise gets what it asks for. Nothing is broken by narrowing the range,
   because the driver has never been upstream.

Where the numbers come from
===========================

The parts taken from traces are labelled as such and not dressed up.
imx681_init_regs[] is 21 register writes whose individual meaning is not known.
Twenty-one, not a thousand-line table, which is the one mercy of this sensor.

What is derived is written down. The link frequency comes from the PLL
configuration visible in the same traces - 19.2 MHz EXCK, PLL2_MUL 303,
PLL2_PRE_DIV 3, giving 1939.2 MHz on the bus and therefore 969.6 MHz per lane -
and the pixel rate follows from that, the lane count and the bit depth. The
gain law and the black level were measured against the sensor rather than read
off a datasheet, because there is no public datasheet for this part.

Still open, and still Andre's to decide
=======================================

The chip-ID read is a single cci_read with no retry. This machine has failed it
twice, once as 0x0081 and once as 0x0000 against 0x0681 - a sensor answering
before it is ready. A retry absorbs that without inventing a longer reset delay.
Andre would rather land the driver as written and add this afterwards, and I
have kept it that way. Unlike the four above, this is not a bug anyone has
reported against the code; it is an improvement he has already deferred.

Worth saying because it may not be this sensor's fault: the ov13858 on the same
machine has now returned a wrong chip id once as well, 0x1000 against 0xd855,
and it reads its id the same unretried way. Two sensors, one board. I would not
draw a conclusion from two events, but a retry looks cheap either way.

The input clock rate is read and logged but never checked against the 19.2 MHz
the register sequence assumes. Same disposition: a follow-up, not a silent edit.

And one that is open in a different sense: 2/3 adds a MAINTAINERS entry naming
Andre as M:, which is the convention for a driver's author but is also an
obligation he has not been asked about, since he is away. It is in the series
because a new driver with no maintainer entry is worse. If he would rather it
named me, or both of us, that is a v3 change and I have put it to him.

Testing, and what it does not reach
===================================

On a Surface Pro 11, front camera, all three patches applied, on a kernel built
from this base rather than backported into a distro one:

  # uname -r
  7.3.0-rc1-imx681-medianext+
  # dmesg
  intel-ipu7 0000:00:05.0: Found supported sensor SONY0681:00 (\_SB.PC00.I2C5.CAMF)
  imx681 i2c-SONY0681:00: supply dvdd not found, using dummy regulator
  imx681 i2c-SONY0681:00: supply dovdd not found, using dummy regulator
  imx681 i2c-SONY0681:00: IMX681 probed successfully: 3844x2640 @ 969600000 Hz link freq
  intel_ipu7_isys.isys intel_ipu7.isys.40: bind imx681 3-0010 nlanes is 2 port is 2

The two dummy-regulator lines are there deliberately. They are the evidence for
change 1: the get succeeds and hands out dummies for the supplies this firmware
does not describe, and probe still completes with the failure now fatal. Had
that reasoning been wrong, the camera would not have come up at all.

  streams                  3844x2640 SGRBG10, 30.01 fps
  analogue gain sweep      65.0 / 65.5 / 67.7 / 84.2 mean, codes 0/300/700/960
                           (a dark room at midnight; the pedestal is near 64)

Change 4 was measured rather than argued, by reading register 0x020E over i2c
while the sensor streams. The stream has to be running: at stream start
__v4l2_ctrl_handler_setup() applies every control in creation order, analogue
before digital, so digital always wins and the clash is invisible. It bites a
live AE loop, which is the case that matters.

                                        before         after
  start                                 0x0100         0x0100
  after digital_gain = 4x               0x0400         0x0400
  after analogue_gain 500 -> 700        0x0100  <--    0x0400
  DIGITAL_GAIN control reads            1024           1024

The last row is the point: the control reports 4x in both columns, and before
the fix the register disagrees with it and nothing says so. Both modules were
built from the same tree and swapped with rmmod/insmod, so the driver is the
only difference between them. v3 carries this code unchanged from v2 and was
re-measured on the hardware after the PM change above, with the same result.

  checkpatch --strict      clean on 2/3 and 3/3; on 1/3 only "does MAINTAINERS
                           need updating?", which 2/3 answers
  make dt_binding_check    passes - CHKDT, LINT, STYLE, and the example
                           extracted and compiled
  build, W=1               imx681.o and ipu-bridge.o, no warnings

And what it does not reach, set out because v2 was caught by exactly the gap
rather than by anything in the list above:

  build configurations     CONFIG_PM is y here and cannot be turned off - on
                           x86 with suspend, PM_SLEEP is def_bool y and selects
                           it. So the !CONFIG_PM path is read rather than
                           compiled, and that is precisely what let v2 through.
                           If anyone has a config where this driver builds with
                           PM off, that is the build I would most like a report
                           from.

  error paths              The regulator get succeeds here because INT3472
                           hands out dummies for dvdd and dovdd, so the fatal
                           path under 1 is reasoned rather than exercised. The
                           same holds for the probe unwind under 3.

  IVSC                     None of INTC1059, INTC1095, INTC100A, INTC10CF,
                           INTC10DE, INTC10E0 or INTC10E1 is present here, so
                           sensor->csi_dev is NULL and ipu-bridge never enters
                           its IVSC paths. That is why the review's finding
                           against ipu-bridge is reported in the answer
                           linked above rather than patched in this series.

  one machine, one part    One Surface Pro 11 and one sensor sample. German
                           Papulindez has the same HID on another machine, but
                           reaches IPU7 through the staging driver, so that is
                           a second path rather than a second test of this one.

  one mode                 3844x2640 SGRBG10 at 969.6 MHz per lane is the only
                           configuration the driver has, and the only one
                           measured.

  no sanitiser run         Not built with KASAN or lockdep. That is a kernel
                           rebuild here rather than a problem, so ask if it
                           would help.

Based on media/next at f9536a8065 ("media: ipu-bridge: Add support additional
link frequency").

German reported this HID as a bug on this list on 3 September and has had no
reply since; he is on Cc here. He reaches IPU7 through the staging driver rather
than ipu-bridge, so if this goes anywhere there is a second machine on a second
path ready to try it.

  https://lore.kernel.org/linux-media/20260903080854.16266-1-germanpapulindez@gmail.com/

Andre Gilerson (1):
  media: i2c: Add Sony IMX681 sensor driver

Sergey Lebedev (2):
  dt-bindings: media: Add Sony IMX681
  media: ipu-bridge: Add Sony IMX681

 .../bindings/media/i2c/sony,imx681.yaml       | 107 +++
 MAINTAINERS                                   |   8 +
 drivers/media/i2c/Kconfig                     |  10 +
 drivers/media/i2c/Makefile                    |   1 +
 drivers/media/i2c/imx681.c                    | 884 ++++++++++++++++++
 drivers/media/pci/intel/ipu-bridge.c          |   2 +
 6 files changed, 1012 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/i2c/sony,imx681.yaml
 create mode 100644 drivers/media/i2c/imx681.c

-- 
2.53.0
cause of the image error in Gnome Snapshot [PATCH v3 0/3] media: Add support for the Sony IMX681
Posted by German 1 week, 6 days ago
From: German Pablo Lindo <germanpapulindez@gmail.com>

I think i found (at least in my system) why Gnome Snapshot dosent work, i dont know if this error is also present on your machine with the patches, so i have organiced this message with commands for you to be able to directly copy-paste them and make tests

There are resolution in wich the cammera works fine like:
qcam --stream "width=1280,height=720" the resolution that uses google meet
qcam --stream "width=640,height=480" resolution on a web camera test online
qcam whithout the stream parameter, which has a cuality of 3836 × 2640

But there are some that experience the violet and lines error i reported o my previous test
qcam --stream "width=1920,height=1080" this one is the resolution gnome snapshot uses
qcam --stream "width=1440,height=810"
qcam --stream "width=960,height=540"

I hope this helps in the development of the driver
Test for [PATCH v3 0/3] media: Add support for the Sony IMX681
Posted by German 2 weeks, 1 day ago
I have tested the three patches mentioned in this email in the linux version 7.3.0-rc2 in order to test if they solve the IMX681 problem

##DEVICE USED
Microsoft surface pro 11
Cachy OS

##WORKING:
The camera is totally functional in apps like qcam and firefox after aplying the apropiate configuration and installing the IPU7 Hal and bin of intel

##NOT WORKING:
For some reason, when i tested the camera on gnome snapshot (version 50, istalled with pacman) the colors where completelly wrong (the whole image had a magenta tint to it in addition with some horizontal magenta and yellow lines and a diagonal line that went from the bottom left cornet to the top rigth corner) and the image was inverted

##MINOR ISSUES
all things reported here are little details barely noticable and dont influence on the usability of the device
- When exposed to a really bright ligth the image starts to gain a green color and blue if the room is almost devoided of ligth
- The cammera has some kind of "Snow" on the image captured
- The image captured tends to be brighter than a normal camera, this error increases the more ligth there is in the room
Re: [PATCH v3 0/3] media: Add support for the Sony IMX681
Posted by Sergey Lebedev 2 weeks, 1 day ago
German,

Thank you - a second machine, which the v3 cover letter named as the
limitation it most wanted lifted. Your report also found something real.

Your inverted image is the driver
=================================

Prompted by it I held a sheet of printed text in front of the camera and
captured one raw frame: /dev/video16, SGRBG10, a crude 2x2 debayer, nothing
clever. The text came out reversed, and flipping the frame horizontally makes
it read correctly. So the sensor output is horizontally mirrored, always -
not rotated, not flipped vertically.

The other half of that result is worth saying: apart from the mirror the frame
is an ordinary photograph. No tearing, no skew, no diagonals, no wrong
geometry.

The cause is in the init sequence, with the author's comment:

    /* Image orientation: H-flip to match Windows AIQB (RGGB native -> GRBG) */
    { CCI_REG8(0x0101),  0x01 },

The flip makes the Bayer order GRBG, which is what the driver advertises and
what Intel's tuning expects, so the format declaration is honest. The geometry
is not declared at all: there is no V4L2_CID_HFLIP, and camera_orientation
says only that the camera faces front, not that frames arrive mirrored.

Which is why one of your applications is right and the other is not. A front
camera is conventionally mirrored by the application. An app that does that
applies a second flip and gets the true scene - qcam, Firefox. An app that
does not shows our output backwards - Snapshot. Both are behaving sensibly;
the driver put them in that position by transforming the image invisibly.

It is fixable above the driver, so nobody is stuck - but only if userspace
knows, and today it cannot find out.

Bug, or acceptable? I would like the list's view
================================================

There is a case for acceptable: the flip is part of the mode, the advertised
Bayer code matches what is really on the bus, and a front camera ends up
mirrored anyway. I lean the other way: your two applications are what it looks
like when a consumer cannot learn about the transform. But that is one machine
and one opinion, and the code is Andre's, who is back on 28 September.

If it is a bug, two shapes:

  1. RECOMMENDED. Keep the flip, expose V4L2_CID_HFLIP defaulting to 1, and
     swap the advertised code with it - SGRBG when set, SRGGB when clear.
     Nothing changes for anyone who does nothing, and the transform becomes
     visible and controllable. imx415 exposes both flips but does not vary its
     code, so the tree is not a clean precedent.

  2. Drop the flip and advertise SRGGB10, leaving mirroring to userspace.
     Cleaner as a driver, but it changes the advertised format - the very
     thing the flip exists to control - and would likely break the setup you
     have working.

1 costs no existing user anything, which is why I recommend it; 2 is the
better driver if the AIQB turns out not to require GRBG, and that is Andre's
to say. Sakari, Dan, Hans - a view either way settles it and I will do the
work.

Two notes alongside that. Nothing here blocks anyone - the camera is usable as
it stands, as your own report shows. This is about making the driver honest,
not about making it work.

And I have an open thread on libcamera-devel about this sensor's delays, still
waiting on a reply:

  https://lists.libcamera.org/pipermail/libcamera-devel/2026-September/061928.html

libcamera consumes precisely what is in question here - sensor orientation and
the flip controls - so a view from that side bears directly on which option is
right. I have not cross-posted this, because it is a kernel-side decision
first, but I will carry the answer across once there is one.

Your other notes
================

The colour cast with light level, the brightness drift and the snow all look
like AE, AWB and tuning, none of which this driver has: it exposes exposure,
blanking, two gains, link frequency, pixel rate and a test pattern, and that
is all.

Two guesses of mine were wrong, so skip them: the capture node's padded line
(7744 bytes for a 3844-pixel row) would draw about nineteen diagonals, not
your one; and imx681_MSHW0520, MSHW0580 and MSHW0580Second are byte-identical
.aiqb files here, so the variant cannot change colour.

One question that is ours
=========================

v3 changed the gain ABI and you tested v3. Up to v2, V4L2_CID_ANALOGUE_GAIN
advertised 0..1020 and quietly drove the digital gain register above code 960,
so one control could reach 256x; in v3 it stops at 960, the 16x the analogue
stage actually does. If your HAL drives only ANALOGUE_GAIN it now has sixteen
times less range - so did brightness behave differently on v1 or v2? If you
only ran v3, saying so is just as useful.

If you are comfortable with it, a

  Tested-by: German <germanpapulindez@gmail.com>

would carry weight: a second machine and a second userspace stack. It would
not be claiming Snapshot works or that the tuning is right.

Thanks again.

Sergey
Test on another machine and userspace for [PATCH v3 0/3] media: Add support for the Sony IMX681
Posted by German 2 weeks, 1 day ago
 Thanks for clarifying the origin of the errors
 I am more than happy to provide the tag
 Tested-by: German <germanpapulindez@gmail.com>
Re: Test on another machine and userspace for [PATCH v3 0/3] media: Add support for the Sony IMX681
Posted by Krzysztof Kozlowski 2 weeks ago
On Thu, Sep 10, 2026 at 01:53:32PM +0200, German wrote:
>  Thanks for clarifying the origin of the errors
>  I am more than happy to provide the tag
>  Tested-by: German <germanpapulindez@gmail.com>

Please use full name for known identities. In general we do not deal
with anonynymous folks.

Best regards,
Krzysztof
Test on another machine and userspace for [PATCH v3 0/3] media: Add support for the Sony IMX681 Thanks for clarifying the origin of the errors I am more than happy to provide the tag Tested-by: German <germanpapulindez@gmail.com>
Posted by German 2 weeks, 1 day ago