Writing UDEV rules to get a SCSI scanner working on Ubuntu

I'm building some Ubuntu 9.04 (Jaunty Jackalope) systems for relatives and using them as a way to get rid of a lot of old hardware that has been taking up space in my office. This includes several old USB, parallel port, and SCSI scanners. SCSI scanners pretty much ruled in the days before USB as they were much faster than parallel ports. However, they were a pain to configure and required heavy (and usually short) cables which made them difficult to fit into your work area. I tested a Microtek ScanMaker E3 (MRS-600E3) and UMAX Vista S8 scanner first. They worked without problems although the former was picky about termination. Unfortunately a Hewlett-Packard ScanJet 6100C (Q2950A) didn't work at all. Checking the kernel messages indicated that it was represented by /dev/sg7 but the permissions were 0660 root:root so sane couldn't access it. Changing the permissions solved the problem but the /dev directory is a virtual filesystem controlled by udev and the changes are lost after reboot. I could just put a chmod comand in /etc/rc.local but that is the wrong way to fix it. A search on launchpad found bug #378989 which describes the problem with this model. I'm not sure if the fault lies with udev or HAL but creating a udev rule is a simple enough way to fix it for now. I'll describe how to create such a rule using this as an example but udev rules can do much more than just change device permissions.

First you need to be root. Either add "sudo" to the beginning of the following commands or switch to a root shell with "sudo su". Next install lsscsi which makes it easy to identify device node assignments:

apt-get install lsscsi

Then run it to get a list of SCSI devices:

lsscsi -g
[0:0:0:0] disk ATA Maxtor 33073U4 BAC5 /dev/sda /dev/sg0
[0:0:1:0] cd/dvd LITE-ON COMBO SOHC-4836V SG$4 /dev/sr0 /dev/sg1
[4:0:5:0] process HP C2520A 3644 - /dev/sg7
[5:0:0:0] disk USB 2.0 Flash Disk 0.00 /dev/sdb /dev/sg2
[6:0:0:0] disk Generic USB SD Reader 1.00 /dev/sdc /dev/sg3
[6:0:0:1] disk Generic USB CF Reader 1.01 /dev/sdd /dev/sg4
[6:0:0:2] disk Generic USB SM Reader 1.02 /dev/sde /dev/sg5
[6:0:0:3] disk Generic USB MS Reader 1.03 /dev/sdf /dev/sg6

Note that the scanner is at /dev/sg7. With this information you can then use udevadm to find out what is known about the device in the udev database and where in hierarchy of systems it lies:

udevadm info -a -p /sys/class/scsi_generic/sg7

Udevadm info starts with the device specified by the devpath and then
walks up the chain of parent devices. It prints for every device
found, all possible attributes in the udev rules key format.
A rule to match, can be composed by the attributes of the device
and the attributes from one single parent device.

looking at device '/devices/pci0000:00/0000:00:1e.0/0000:01:01.0/host4/target4:0:5/4:0:5:0/scsi_generic/sg7':
KERNEL=="sg7"
SUBSYSTEM=="scsi_generic"
DRIVER==""

looking at parent device '/devices/pci0000:00/0000:00:1e.0/0000:01:01.0/host4/target4:0:5/4:0:5:0':
KERNELS=="4:0:5:0"
SUBSYSTEMS=="scsi"
DRIVERS==""
ATTRS{device_blocked}=="0"
ATTRS{type}=="3"
ATTRS{scsi_level}=="3"
ATTRS{vendor}=="HP "
ATTRS{model}=="C2520A "
ATTRS{rev}=="3644"
ATTRS{state}=="running"
ATTRS{timeout}=="0"
ATTRS{iocounterbits}=="32"
ATTRS{iorequest_cnt}=="0x8"
ATTRS{iodone_cnt}=="0x8"
ATTRS{ioerr_cnt}=="0x1"
ATTRS{modalias}=="scsi:t-0x03"
ATTRS{evt_media_change}=="0"
ATTRS{queue_depth}=="2"
ATTRS{queue_type}=="none"

looking at parent device '/devices/pci0000:00/0000:00:1e.0/0000:01:01.0/host4/target4:0:5':
KERNELS=="target4:0:5"
SUBSYSTEMS=="scsi"
DRIVERS==""

looking at parent device '/devices/pci0000:00/0000:00:1e.0/0000:01:01.0/host4':
KERNELS=="host4"
SUBSYSTEMS=="scsi"
DRIVERS==""

looking at parent device '/devices/pci0000:00/0000:00:1e.0/0000:01:01.0':
KERNELS=="0000:01:01.0"
SUBSYSTEMS=="pci"
DRIVERS=="aic7xxx"
ATTRS{vendor}=="0x9004"
ATTRS{device}=="0x7178"
ATTRS{subsystem_vendor}=="0x0000"
ATTRS{subsystem_device}=="0x0000"
ATTRS{class}=="0x010000"
ATTRS{irq}=="22"
ATTRS{local_cpus}=="ffffffff,ffffffff" ATTRS{local_cpulist}=="0-63"
ATTRS{modalias}=="pci:v00009004d00007178sv00000000sd00000000bc01sc00i00"
ATTRS{enable}=="1"
ATTRS{broken_parity_status}=="0"
ATTRS{msi_bus}==""

looking at parent device '/devices/pci0000:00/0000:00:1e.0':
KERNELS=="0000:00:1e.0"
SUBSYSTEMS=="pci"
DRIVERS==""
ATTRS{vendor}=="0x8086"
ATTRS{device}=="0x244e"
ATTRS{subsystem_vendor}=="0x0000"
ATTRS{subsystem_device}=="0x0000"
ATTRS{class}=="0x060400"
ATTRS{irq}=="0"
ATTRS{local_cpus}=="ffffffff,ffffffff"
ATTRS{local_cpulist}=="0-63"
ATTRS{modalias}=="pci:v00008086d0000244Esv00000000sd00000000bc06sc04i00"
ATTRS{enable}=="1"
ATTRS{broken_parity_status}=="0"
ATTRS{msi_bus}=="1"

looking at parent device '/devices/pci0000:00':
KERNELS=="pci0000:00"
SUBSYSTEMS==""
DRIVERS==""

Note that the DRIVERS=="aic7xxx" indentifies the Adaptec AHA-2940 SCSI card. All of this data can be referenced by a udev rule to identify when and how to manipulate the device. That is what udev does - run everything through a list of rules, matching or excluding attributes as specified by a rule, then performing an operation when the conditions of a rule is met. The manual for udev is at /usr/share/doc/udev/writing_udev_rules/index.html and it gives many good examples of what you can do. In this case the scanner device needs different permissions and group ownership so that users can access it with Xsane. Most of the rules included with packages are in /lib/udev but local rules can be added to /etc/udev/rules.d and they can override existing rules. There is a file name standard for the rule files (see the README in the directory) - they always start with a number (which indicates priority) and end with ".rules". My rule file is "/etc/udev/rules.d/45-scsi-scanner.rules", owned by root and in group root with 0644 (rw-r--r--) permissions. You have to reboot to make it active. This is what it contains:

# permissions for HP ScanJet 6100C SCSI scanner
SUBSYSTEM=="scsi_generic",ATTRS{vendor}=="HP",ATTRS{model}=="C2520A", NAME="%k", SYMLINK="scanner%n", MODE="0660", GROUP="scanner"

So what does this all mean? First the SUBSYSTEM keyword says it only applies to devices in the "scsi_generic" subsystem (as per the first few lines that udevadm reported). The "==" is a comparison operator. Next the ATTRS{vendor} keyword specifies that an attribute named "vendor" in the subsystem (or any parent subsystem) has to have a value of "HP" (which the SCSI module reports via the SCSI card). Then the ATTRS{model} keyword tells udev to look in the same subsystem that matched the vendor for a model attribute that matches "C2520A". If it finds one, and since there are no other comparisons specified, then the rule matches and the rest is processed. NAME is the keyword for setting the device node name (sg7 in this case) and the %k is a string substitution operator that udev will expand to the original name assigned by the kernel (again sg7). The "=" is the assignment operator. So this part of the rule sets the NAME assignment key to the original "sg7" effectively keeping the default device node "/dev/sg7" as is. The SYMLINK keyword creates symlinks to the default device node. The %n operator is expanded by udev to the kernel number of the device (the 7 in sg7). The resulting symlink will be scanner7 in this case and if the default node changes due to a SCSI device being added or removed the symlink will change to match (scanner5 for sg5, etc.) For the scanner rule it is for convenience only as a device named "scanner" is easier to figure out than "sg", especially when trying to do user support over the phone. The MODE just sets the permissions in octal and GROUP assigns a specific group membership of "scanner".

When this rule is activated, /dev/sg7 will be root:scanner with rw-rw---- permissions and a /dev/scanner7 symlink will also be created that points to it. For the user to access the scanner they need to be in the scanner group. If the scanner group doesn't exist (not in /etc/group) then you can add it with:

addgroup --system scanner

This will dynamically create a system group somewhere in the range of 100-999. Any users added to the group need to relogin for it to take effect.

Introducing Winesharer - so pre-alpha it doesn't even work

About a year ago I was setting up a Ubuntu system for a family of non-technical users. They like to play games and one of their favorites is Diablo II. It works well on Wine if you set it to use Glide and install a Glide wrapper. The game's copy protection is properly supported by Wine but it's not necessary as Blizzard removed the CD check in the recent patches.

There are two problems with installing Windows applications like Diablo II for multiple users. First, because of the isolation of user accounts on *nix systems, you have to login and repeat the install process for each account. Second, each installation after the first wastes disk space and with Diablo II it's several gigabytes, especially if you have a lot of mods.

It's possible to manually copy the first installation and edit the Wine registry, menu entries, and fix symlinks, but it's tedious. So I began messing around with some shell scripts to automate the process. I'm not an expert with shell scripts but I improved with time and some help from my LUG-mates. After several false starts I got a basic script functioning. That solved the first problem but not the space issue.

After messing around with some LiveCDs, I got the idea to try to share the wine directory with a union mount. First I tried FunionFS but it had several bugs that prevented it from working (like not being able to change an existing file). So I switched to Aufs. It worked but it can't be run by a user as it requires root permission to mount. To keep it easy for users I had to use pam_mount and mount it at login. I added the ability for the script to export a sample mount entry for pam_mount.conf.xml to save time.

I then got the idea to add handling for separate Wine directories for each application (like CrossOver Bottles). That brought up another issue, the desktop menu entries for Wine's utilities like winefile and winecfg. I needed to add duplicate entries with different WINEPREFIX settings and associate them with each application. I came up with a primitive solution for locating the entries and duplicating them in an alternate location as a submenu below the primary application's menu (Wine > Programs > (application) > Wine Utils).

I then got another idea - application merging. One problem with games is that there are a lot of third-party mods and other customizations for them. There are also a lot of updates. This requires editing of configuration files, extracting files from archives, and file management. These can't always be done easily with Linux tools. One problem is patches for older games are often in zip files. The contents are intended to overwrite existing files but sometimes the filenames have different case. If you extract them with a native Linux application you end up with duplicates instead of overwrites. The other problem with Linux tools is that they always give a "/" or "/home/user" oriented view when the user expecting C:, especially when following an online instructions for installing a patch or mod. The concept of application merging is simple - select some utilities that have minimal dependences, install to a separate Wine directory, then add it as an Aufs branch to the mount for each main application. You install each one once and share it but then it can be used within every other Wine application directory (and behaves as if it was installed in each) without wasting much storage space. Wine's registry files are text so entries for a merging application can be duplicated with just diff and patch.

Then I realized this had another major benefit for game mods - it would be possible to install multiple, even conflicting mods, and mount them as separate Aufs mount points at the same time. This is especially useful for games that don't have integrated mod management. Using Diablo II as an example, you would install and update it, share it with wineappshare.sh, then mount it via Aufs with a new read/write directory branch. Then install a mod like MedianXL (which ends up in the read/write branch). Then unmount the directory, move and share the read/write directory as a new read-only branch for MedianXL and mount it on top of the Diablo II directory with a new read/write branch. If you mount the Diablo II directory again using a different read/write branch, you can run regular Diablo II and the MedianXL version at the same time. Effectively they are in separate "bottles" but share the bulk of the install in read-only branches so there is little additional overhead.

Of course the tricks don't stop there. You can imagine putting the shared parent branches on a compressed volume to save space, mount it on a server via NFS, and use pam_mount to mount the read/write branches on a USB drive on the client. Imagine the possibilities for a gaming cafe system.

At this point I knew Winesharer would revolutionize Windows gaming on Linux. Just as soon as I finish writing it. Then perfecting it. Then documenting some really complicated examples. And make some awesome video demonstrations. And write the book. Then push it out of the code cave and bask in the glory. At least that was the plan about a year ago before life and reality got in the way. So now I'm down to the old "release early and often" process which completely eliminates the "shock and awe" value. At least this way it may have an impact before Wine is forgotten due to lack of interest in running legacy Windows applications and everyone switching to GNU/Hurd.

Winesharer consists of three scripts:

wineappshare.sh - strips out user-specific directory links, tracks down related icons and XDG menu entires, and copies the Wine directory (bottle) to a shared location - /srv/wine by default. The hardest part was finding the menu entries (*.desktop) as Wine doesn't keep track of them. I had to do a linear search by grepping for matching WINEPREFIX values then calculating what the matching menu directory (*.menu) path would be. It also doesn't help that the "Icon=" references can specify icon file extensions with ".png", ".xpm", or not at all.

winemergeprep.sh - makes registry diffs and a list of modified files for "mergeable" utilities. The file listing excludes any unchanged/unused files like the fake DLLs that Wine adds in the System directory. It is run twice - after initial creation and configuration of the Wine directory and before the target application is installed, and again after installation and configuration of the application. The applications I was using are 7-Zip, xplorer², FontPage, IPaddress, SciTE (or EditPad Lite), @icon sushi, and Dependency Walker. Some of these are only for debugging. After prep the mergeable application directory is shared with wineappshare.sh like other applications and wineappinstall.sh performs the special handling of their branches and patches when other applications are installed for users.

wineappinstall.sh - where things got ugly. Setting up the read/write directory and creating the template pam_mount entry for Aufs mounting was easy. So was copying over the icons. Recreating symlinks between the user's profile directories (My Doucments > ~/Documents, etc.) was a lot more complicated. I wanted to do it correctly by following the XDG Base Directory Specification, first by checking for a local (user) configuration, then the system-wide defaults, then look for commonly-used defaults, and finally just defaulting to ~. I got that part sort of working. The final problem was trying to merge the shared application menu entries, the "mergeable" applications entires, and any existing entries without damage. Doing this in an orderly (and deterministic) fashion is difficult and shell scripts aren't great for text processing. That's where I left off.

The scripts all require the Wine directory name and it's assumed to be in ~. For example, specifying Diablo II's directory would just be ".wine-Diablo_II". Note that spaces should not exist in the directory name. The Winesharer scripts handle them but others, like Dan Kegel's winetricks, had trouble with them. Second, the scripts search through the registry for the username of the installer in order to change it to the target user's name later. Because of this it needs to be globally unique (in ALL Windows applications) so the scripts don't change something that is not related to the user. I had a Wine administration account named "wineadmin" which should be safe as long as there isn't any client/server wine (the drinking type) management applications that use the same keyword or value in the registry. The sharing directory is in /srv to comply with v2.3 of the Filesystem Hierarchy Standard. I was using a "_rw" suffix for the read/write branch directories.

What's next? Nothing. This was intended to be a one week feasibility study but suffered from a ridiculous amount of feature creep. Between the earlier draft scripts and command-line tests I know it's possible to do but text processing in shell scripts is tedious and I don't have the time to finish it. The scripts are ugly, broken, and can't handle all possible problems. I do like cats so I made it a point to eliminate cat abuse but I left a lot of dysfunctional grep|sed marriages since reducing them is time consuming and the extra processes give my idle Phenom cores something to do. There's a lot of arrays and case statements. I didn't follow any column limits either. Because this was a work-in-progress I also discarded the Unixy notion of minimal feedback - my scripts write entire novels to the terminal. There are a lot of comments, especially in the unfinished portions of wineappinstall.sh (which is guaranteed to not work). I didn't even begin implementing the integrated multimedia help styled after 0verkill. Instead, this pre-alpha work includes a bonus pack of bugs. I don't think the scripts can fail in such a way as to wipe out your filesystems and install Vista but I'm not guaranteeing they won't either. This mess was developed on Ubuntu 8.04 (Hardy Heron).

My goal with this project is to inspire others to implement a more robust (not to mention functional) solution incorporating these ideas. A user space union filesystem like FunionFS would be more convenient than Aufs but I don't know of any alternatives. I think that having submenus for mergeables in each application menu is ugly. A front-end utility for dynamically setting WINEPREFIX and launching them would be better. One problem I thought of but don't know how to handle are applications that require registration keys at installation instead of first-run. For some, their registration keys can be purged from the registry and they will prompt for a new key when executed again. Others won't and may lock-out and refuse to run even with a valid key, requiring a full reinstall.