When a driver challenges the kernel’s assumptions
5 hours later:
mglock: displaylink rev. eng have a mailing list? where? erm, yes. https://lists.sourceforge.net/lists/listinfo/displaylink-devel mglock: thanks hmmm, glancing at the library it is not that useful disappointing
2 hours later:
mein gott! libdlo is ugly ERR_GOTO(function(args)) ...
and 2 more hours later:
i've setup another mail to displaylink asking them why they didn't include the compression stuff in their library :-)
On the same day, on the displaylink-devel list:
Date: 2009-05-15 13:59:30 From: Florian Echtler To: displaylink-devel mailing list Subject: Re: [Displaylink-devel] News from Displaylink > Sigh, pretty disappointing. LGPL licensed first and then it doesn't > contain the compression stuff. What do the think how people can > write an usable X.Org driver with this? I gonna ask them. All right, this is getting a bit ridiculous. Looking in dlo_usb.c, they labeled the "set encryption" request as "select channel", and the "null-key" for disabling encryption is called STD_CHANNEL. Okay, maybe just terminology. However, in dlo_data.h, there's suddenly always an DLO_MODE_ENABLE_* sequence (never seen that before), and the DLO_MODE_DATA stuff looks totally random. Hey, look, in mode_select(), it's always sending the ENABLE sequence to the "select channel" command, then the binary blob and then DLO_MODE_POSTAMBLE. Hmm, this POSTAMBLE looks just like the STD_CHANNEL default key! Oh, what a surprise, the register sets for the modes are _still_ encrypted - never mind that we can decrypt this since Christmas last year. Displaylink, I really don't get this. I'll have a closer look at the way the stride registers are implemented; this is one of the smaller parts which I still would like to figure out, but in general, this library is absolutely useless to the wider opensource community. It's obviously designed for some embedded shops which want to use it for non-realtime stuff like LCD signs etc., but it's just ridiculous to still try and obfuscate parts of an opensource library. Florian -- 0666 - Filemode of the Beast
(emphasis mine)
Two more weeks pass, during which Florian Echtler continues his reverse
engineering efforts and figures out the compression scheme used by the
DisplayLink chip.
for those who are interessted in the displaylink stuff ... the compression has been cracked. florian echtler did a library and showed it to the displaylink guys ;-) http://lists.freedesktop.org/archives/libdlo/2009-May/000092.html http://floe.butterbrot.org/external/tubecable-0.1.tar.bz2 ~mglocker/dldemo2_mglocker.tar.gz if you want to have a kind of C-style demo for the compression.
Minor fixes to the udl driver occur on june 1st, then there is not much
activity until Marcus resumes working on it, in order to add compression
support.
On august 14th, he mentions the need for a compression table which could be
stored on the filesystem, rather than in the kernel image:
anyone mind if we would keep a 300kB huffman table in /etc/firmware? that's not really firmware... can't you generate the table at runtime? is this for udl evilness? jup, for the huffman pixel difference compression. i could store the table into an *.h file, but i'm not sure if this is what we want. i don't think we can "generate" the table. but yes, although it's not technically firmware, it's loadable data, so why not yeah. it's probably worth an exception in that case. i can draw some compressed stuff already, but some issues left to fix.
And on august 25th,
compression support
is added to the driver.
As more and more developers are trying the driver on as many platforms as
possible, we start hitting situations where heavy screen adjustments would
cause the output to stall, or some display updates to get lost.
Investigating these showed that the device was overflowed with requests, and
needed time to process all the pending operations until further requests could
be made.
This was not very different from a serial line link, where the serial chip has
a small (and sometimes nonexistent) FIFO queue, and no more characters can be
transmitted when it gets full, until the characters get transmitted over the
wire.
For serial links, the kernel handles this nicely, by forcing writers to sleep
when the FIFO gets full, and waking them when space becomes available in the
FIFO.
But for display terminal emulations, there was no such thing, because it was
assumed that every display operation (output a character, draw the cursor shape,
scroll the display in any direction) could be done without having to wait.
And in fact, until the DisplayLink driver came into use, this was the case:
display drivers either had full access to the display memory and could do the
required changes directly in memory or, as e.g. for the
Vax
gpx driver
[which I will write the story of soon…], by instructing the graphics
controller to perform these changes.
But here, over the USB bus, bandwidth has to be shared with all the other USB
devices on the controller, and the USB controller itself might not be able to
queue enough requests for the udl needs – in other words, this is similar to
the serial FIFO.
The graphics display being, from the kernel point of view, a special case of
serial line, we could benefit from the FIFO behaviour, even though this had
never been the case.
So a cheap and easy way to solve the “display gets overflowed with requests”
problem would be to simply let the display operation routines (draw the cursor,
paint a character…), which until now where `void` operations, return a value
letting their caller know whether they had been able to perform the requested
operation or not. The DisplayLink specific implementation of these requests
could then return failure when the device FIFO is full, and let the upper
layers know that they need to wait and force the process currently writing
to the display, to sleep.
This would be a large, but mostly mechanical, change to all the display drivers
in the OpenBSD source tree.
On august 30th, I suggested going in this direction:
Date: Sun, 30 Aug 2009 20:42:29 +0000
From: Miod Vallat
To: private mailing list
Subject: wsdisplay asynchronous processing
After discussing udl(4) behaviour with mglocker@, we came to the
following thoughts which I believe are worth sharing.
wscons - and its wsdisplay output part - has been written 10 years ago
with tga(4) and vga(4) style bitmapped devices in mind, with synchronous
operations.
This assumption was true, until we started to support video devices on
which the video memory is not directly accessible. Examples of this are
cfxga(4), gpx(4) on vax, and udl(4).
The first two such drivers have been coerced to fit the synchronous
model, because sending commands to the hardware was fast and did only
involve the device registers.
udl(4), however, is completely alien to this. Commands are sent with USB
pipes, there might be other devices on the USB bus, and processing is
asynchronous in nature.
Now, if you look at wsdisplay, there are two conditions upon which
wsdisplay will want to update the display contents:
- userland writes to the wsdisplay terminal. wsdisplay is invoked from
the tty layer, and has a process context, so it can sleep.
- kernel writes to the console device. wsdisplay can not assume
anything.
So if we are able to flag devices such as udl(4) as not being able to be
a console output device, then it will know that it can sleep at the
wsdisplay level.
Why am I talking about sleeping? Because udl(4) is currently the only
wsdisplay(4) driver unable to behave synchronously. In order to complete
its operation, it needs to get fifo resources and whatnot, and if none
of them are available, there won't be any new resources available until
the usb stack has a chance to run: udl(4) needs to sleep.
Now if udl(4) is allowed to sleep, and the driver<->wsdisplay interfaces
are modified so that a driver can return EAGAIN if it is not a console
device, then wsdisplay(4) can sleep. Problem solved.
and since the best way to figure out if such a proposal is worth doing is to
write a proof-of-concept, the next day I shared a diff with Marcus.
But that diff was larger and trickier that what my previous message implied.
The reason for that, is that the graphics terminal is not simply a “line printer
on a CRT”.
(and, writing this in 2025, I suppose some of my younger readers will not be
familiar with
Cathode Ray Tube
displays)
An important improvement of CRT displays above line printers is that most CRT
terminals have an addressable cursor position; this opens the road towards
fancy displays (nowadays with colours!), first to implement
IBM 3270 terminal
emulators, later for more generic needs, thanks to the
termcap and/or
terminfo abstractions,
and eventually the
curses library.
The OpenBSD workstation console terminal emulation tries to mimic two well-known
console devices, the
Digital VT220 terminal
(minus several seldom used features difficult to implement cleanly,
such as double-width characters) on most platforms, and the Sun console
(with extra features such as colour) on
sparc and
sparc64 ports.
(I tried to have the
luna88k port, which
defaults to black on white display like the Sun systems,
to use the sun terminal emulation, but got outpowered).
Both these terminal emulations have in common that they recognize specific
escape sequences, e.g. to change the cursor position, change output colours, or
other specific operations. Also, simply outputting a regular character expands
to at least three display operations: hide the cursor, draw the character, and
draw the cursor in its new position; and if the character had been the last on
the last line, the display needs to scroll, which involves one more operation.
If display drivers were able to fail any operations, as allowed in my proposal,
we could end up with a particular output operation processed only partly.
But the tty layer in the kernel has no way to know that a character has been,
sort of, “partially output”. In this layer, a character has either been output,
or needs to be output, and nothing in-between. Any finer-grained state needs to
be maintained in a lower layer, such as the vt220 emulation code.
So, in addition to allowing display drivers to fail operations, I wrapped the
terminal emulation processing into what I called an “abort state”, remembering
at which point we encountered a failure, so that attempting to output a
character would skip the operations which had succeeded already.
With my changes, a character output would only be “validated” if all the display
operations it would cause had been successful. If not, the process writing to
the display would be forced to sleep, until the display driver reports it is
able to process requests again. Processing of the display operations would note
which operations had been successful, so as to skip them and only perform the
operations which had failed or had no chance to start earlier.
Date: Mon, 31 Aug 2009 22:00:27 +0000 From: Miod Vallat To: Marcus Glocker Subject: early wscons `ok to stall' diff This should be enough for an i386 or amd64 kernel to compile. Many frame buffer drivers still need to be modified due to interface and prototypes changes. How does it work? Well, this adds an error path from the display driver to the tty layer. So we have this path: 1. void wsdisplay_emulops routines now return values. The driver implementing them can return 0 if it did the work, or nonzero (preferrably EAGAIN) if it couldn't. Note that generic rasops routines, accessing frame buffer memory, never fail and always return zero. 2. the return values of the emulops are now tested in the wsdisplay emulation code. This is the horrible part with a lot of changes, which will need careful review. 3. I designed this so that, when the emulops return an error, the emulation state machine is moved back to a sane state, allowing the operation to be tried again later. This involves undoing logical cursor moves, and other internal state changes. Note the code currently assumes an operation involving several emulops can fail in the middle and be restarted from the beginning - this is wrong, e.g. when scrolling the tty, since we copy rows and then clear the bottom row. If the copy works but the clearing fails, we'll restart with the copy. I am aware of this and working on an `abort state' part of the emulation state machine (which is currently the FALSE_ABORT bit in the per-emulation flags, but needs to move into its own thing). (This is why this is an alpha diff...) 4. The error condition detected by the emulation causes the tty write to abort early. The MI wsdisplay code will now know how many chars have been processed, and if it detected an early abort, it will not try to write more characters, even if there are any left in the tty queue. 5. The same function already has logic to schedule a timeout if more tty data is pending. This timeout will try to feed the driver more work, but until it fires the driver can hopefully get some interrupts and gather resources to do so. What is left to do: 1. Update all MD frame buffer drivers (mechanical). 2. Finish the abort state design and correctly recover from a partial operation. 3. Update udl to return EAGAIN in the emulops. Known problems which won't be fixed soon: 1. Some operations do not come from the tty layer, but by keyboard events (emulation reset sequence), ioctl or timeouts (screen burner). We do not necessarily have a process context there, so sleeping is not an option. I need to extend some interfaces for the affected routines to know whether the caller can recover automatically (tty context), or not, and if not, whether it's ok to sleep or not. 2. There is no way, yet, for the driver which has returned EAGAIN to cause the tty timeout to be triggered earlier (i.e. as soon as it regains resources). I'll think about it eventually. Miod (and now time for some zzz)
The next day, I received the best testimonial ever for a diff:
hi from udl over wscons with initial EAGAIN support :-) miod is evil.
After more testing, on september 1st, a new version of that diff was shared:
Date: Tue, 1 Sep 2009 20:19:02 +0000
From: Miod Vallat
To: private mailing list
Subject: Re: wsdisplay asynchronous processing
In case people are interested, I have a monster diff (which will be
split in 4 different pieces), which implements error path from the
display drivers up to the tty layer, so that the driver can cause tty
output to stop if it is overflowed.
The tty layer (well, the wsdisplay tty code) will then nicely recover
from this, and everything is fine.
This diff has the disavantage of adding about 1KB of code to the kernel
(for kernels with vt100 emulation), so this might be a problem for the
floppies. And there is no way to disable this if SMALL_KERNEL. Unless
you want to dive in your own sea of macro hell filled with sharks.
Due the large size of the diff (about 200KB, affecting 51 files), I will
not append it to this mail. People interested in it can find it in
cvs:~miod/wscons-stall2.vari
Note that, for it to be really useful on udl(4) devices - since they are
the reason for this work - you need another diff from mglocker@, adding
the necessary EAGAIN code in udl(4).
$ wc wscons-stall2.vari
7017 26187 194603 wscons-stall2.vari
Miod
The monster diff started with the following description:
This diff is large because many frame buffer drivers need to be modified due to interface and prototype changes. How does it work? Well, this adds an error path from the display driver to the tty layer. So we have this path: 1. void wsdisplay_emulops routines now return values. The driver implementing them can return 0 if it did the work, or nonzero (preferrably EAGAIN or EINTR) if it couldn't. Note that generic rasops routines, accessing frame buffer memory, never fail and always return zero. 2. the return values of the emulops are now tested in the wsdisplay emulation code. This is the horrible part with a lot of changes, which will need careful review. 3. I designed this so that, when the emulops return an error, the emulation state machine is moved back to a sane state, allowing the operation to be tried again later. This involves undoing logical cursor moves, and other internal state changes. Note there might be bugs in the undoing so far, I need to review this carefully. And test too (-: There are comments in wsemulvar.h trying to explain how I keep track of failures occuring in the middle of a `character' (from the tty layer point of view) output. 4. The error condition detected by the emulation causes the tty write to abort early. The MI wsdisplay code will now know how many chars have been processed, and if it detected an early abort, it will not try to write more characters, even if there are any left in the tty queue. 5. The same function already has logic to schedule a timeout if more tty data is pending. This timeout will try to feed the driver more work, but until it fires the driver can hopefully get some interrupts and gather resources to do so. Note that it is not necessary to implement a faster output resume path (e.g. if the driver gets an interrupt and frees resources), as the timeout will run only 8ms later (1/128 second). Keep in mind the human persistance of vision is about 1/25 second, so in the blink of an eye the wsdisplay code can resume stalled output several times. Problems left with this code: 1. You may notice resetop() does not check for emulops failure. This because this is an out-of-tty-layer processing (but ioctl issued to the tty device). I know how to make it able to recover, but this will need a few more emulops prototypes changes, and I would like to keep the focus of this diff minimal (har, har), i.e. trying to only address one problem. A later diff will address that area. 2. The same comments apply to the screen burner code. Again, I have plans for this. How I intend to split this work in individual commits: 1. internal wsemul changes to change the state machine functions from returning u_int to returning void (and updating emul state structs directly), so that they can later be changed again to return errors. (no functional change, but little code growth) 2. emulops prototype changes (and all the rasops / driver part of this diff). (again, no functional change, but little code growth) 3. wsemul_ops change of output() to return the number of chars consumed, and the corresponding logic in wsdisplaystart(), with the emulation code returning the number of chars it has been given (again, no functional change, but little code growth) 4. the error path handling in the emulation code, i.e. the evil part of this diff.
Of course, testing exposed a few bugs in that diff, which led to a new version:
Date: Wed, 2 Sep 2009 16:17:06 +0000
From: Miod Vallat
To: private mailing list
Subject: Re: wsdisplay asynchronous processing
New diff fixing a few bugs in the previous diff (description of changes
at the head of the new diff).
cvs:~miod/wscons-stall3.vari
$ wc wscons-stall3.vari
7074 26494 196861 wscons-stall23vari
Miod
And while writing this, I am dissatisfied with past me. It was not a wise
idea to put the details in a file which is now long gone, and I should have
put them both in the file and in the email. But I did not think that future me
would want to tell this story, years later.
Fortunately, it turns out that I had sent that diff directly to Marcus minutes
later:
Date: Wed, 2 Sep 2009 16:13:51 +0000 From: Miod Vallat To: Marcus Glocker Subject: latest wscons diff This is diff #3. Changes since last diff: - rearrange changes to wsdisplaystart() so as not to introduce a new goto. - minor simplification in wsemul_sun when backing out state changes because of failed scrollup operation. - rearrange some double-wide array updates in vt100 to be able to correctly recover from operations aborted in the middle. This double width feature is something noone uses anyway. - wrap the COPYCOLS and ERASECOLS operations in vt100 within WSEMULOP so that they will not be reissued by mistake if failure occurs after they're done. This ought to have been in the previous diff but I forgot to do this.
And some time later:
Date: Thu, 3 Sep 2009 22:00:06 +0000
From: Miod Vallat
To: Marcus Glocker
Subject: Re: udl
Even better, with a diff that still compiles after the untested last
minute change.
This is diff #4.
Changes since last diff:
- removed udl.c changes, get them from mglocker@
- fixed an unitialized variable in wsemul_vt100_output() causing cursor
image display to sometimes be skipped.
- extended the abort state with four different states (ok, failed to display
the cursor image, failed to jump scroll, failed a regular operation) instead
of two (ok/fail) and having the `fail cursor' a particular state of failure.
This allows me to make things a bit more clean (arguably) and two fix two
important bugs:
+ after a regular failure, jump scroll would be attempted before the
other operation; if the failure had happened after N operations we would
then skip N operations during the jump scroll (usually causing the
scroll not to happen, so lines would be overwritten instead of
scrolling).
+ after a jump scroll failure, we need to retry it with the same number
of lines as the failed operation, so that the copy/erase parts of the
scrolling operation are consistent with each other.
Eventually Marcus confirms it works:
From: Miod Vallat To: Marcus Glocker Subject: Re: udl > This diff works pretty fine for me! I can't spot any bugs anymore yet, > even when running in a screen session with crazzy apps like irssi and > mutt. Excellent.
Having been able to wrap these operations in specfic macros to hide the
note-and-restart logic, this allowed installation media kernels (which would not
embed the DisplayLink driver) to use the previous “nothing can fail” logic, in
order not to grow these kernels and still allow them to fit on 3″½ floppy
installation media.
This work eventually got commited on september 5th,
in
multiple
steps
and
then
some…
…and udl made use of it
immediately.
On september 11th, this allowed to nicely fix a
diresome situation.
And since I am only human, a
small bugfix
was needed on the 14th.
After these changes, there have been no problem reports with DisplayLink
devices.
Later this month, an X server, based upon the “damage” extension which describes
areas which need to be redrawn, was also added to the OpenBSD source tree, and
support for the DisplayLink devices was now complete.
This allowed “serial console-only” platforms such as the
armish
and
landisk
ports to use a graphics console and run an X server.
And even if there had been no X server support, the DisplayLink driver forced
the console code to face new challenges and solve them in a way which will
benefit future drivers.
Today, DisplayLink still exists as part of
Synaptics,
and their most recent chips are supported neither by Linux nor by OpenBSD with
open source drivers. Synpatics provides a binary driver for Ubuntu, which to
this day hasn’t been reverse engineered, yet.
The usefulness of these devices has apparently gone away, people do not seem to
be interested by these devices anymore those days.
