Thursday, March 27, 2014

Viewing the Xorg.log with journalctl

Those running Fedora Rawhide or GNOME 3.12 may have noticed that there is no Xorg.log file anymore. This is intentional, gdm now starts the X server so that it writes the log to the systemd journal. Update 29 Mar 2014: The X server itself has no capabilities for logging to the journal yet, but no changes to the X server were needed anyway. gdm merely starts the server with a /dev/null logfile and redirects stdin/stderr to the journal.

Thus, to get the log file use journalctl, not vim, cat, less, notepad or whatever your $PAGER was before.

This leaves us with the following commands.

journalctl -e _COMM=Xorg 
Which would conveniently show something like this:
Mar 25 10:48:41 yabbi Xorg[5438]: (II) UnloadModule: "wacom"
Mar 25 10:48:41 yabbi Xorg[5438]: (II) evdev: Lenovo Optical USB Mouse: Close
Mar 25 10:48:41 yabbi Xorg[5438]: (II) UnloadModule: "evdev"
Mar 25 10:48:41 yabbi Xorg[5438]: (II) evdev: Integrated Camera: Close
Mar 25 10:48:41 yabbi Xorg[5438]: (II) UnloadModule: "evdev"
Mar 25 10:48:41 yabbi Xorg[5438]: (II) evdev: Sleep Button: Close
Mar 25 10:48:41 yabbi Xorg[5438]: (II) UnloadModule: "evdev"
Mar 25 10:48:41 yabbi Xorg[5438]: (II) evdev: Video Bus: Close
Mar 25 10:48:41 yabbi Xorg[5438]: (II) UnloadModule: "evdev"
Mar 25 10:48:41 yabbi Xorg[5438]: (II) evdev: Power Button: Close
Mar 25 10:48:41 yabbi Xorg[5438]: (II) UnloadModule: "evdev"
Mar 25 10:48:41 yabbi Xorg[5438]: (EE) Server terminated successfully (0). Closing log file.
The -e toggle jumps to the end and only shows 1000 lines, but that's usually enough. journalctl has a bunch more options described in the journalctl man page. Note the PID in square brackets though. You can easily limit the output to just that PID, which makes it ideal to attach to the log to a bug report.
journalctl _COMM=Xorg _PID=5438
Previously the server kept only a single backup log file around, so if you restarted twice after a crash, the log was gone. With the journal it's now easy to extract the log file from that crash five restarts ago. It's almost like the future is already here.

Update 16/12/2014: This post initially suggested to use journactl /usr/bin/Xorg. Using _COMM is path-independent.

Fedora 21

Added 16/12/2014: If you recently updated to/installed Fedora 21 you'll notice that the above command won't show anything. As part of the Xorg without root rights feature Fedora ships a wrapper script as /usr/bin/Xorg. This script eventually executes /usr/libexecs/Xorg.bin which is the actual X server binary. Thus, on Fedora 21 replace Xorg with Xorg.bin:

journalctl -e _COMM=Xorg.bin
journalctl _COMM=Xorg.bin _PID=5438
Note that we're looking into this so that in a few updates time we don't have a special command here.

Tuesday, March 25, 2014

CyPS/2 Cypress Trackpad and firmware-based button emulation

For a longer story of this issue, please read Adam Williamson's post. The below is the gist of it, mostly for archival purposes.

The Dell XPS13 (not the current Haswell generation, the ones before) ships with a touchpad that identifies as "CyPS/2 Cypress Trackpad". This touchpad is, by the looks of it, a ClickPad and identifies itself as such by announcing the INPUT_PROP_BUTTONPAD evdev property. In the X.Org synaptics driver we enable a couple of features for those touchpads, the most visible of which is the software-emulated button areas. If you have your finger on the bottom left and click, it's a left click, on the bottom right it's a right click. The size and location of these areas are configurable in the driver but also trigger a couple of other behaviours, such as extra filters to avoid erroneous pointer movements.

The Cypress touchpad is different: it does the button emulation in firmware. A normal clickpad will give you a finger position and a BTN_LEFT event on click. The Cypress touchpads will simply send a BTN_LEFT or BTN_RIGHT event depending where the finger is located, but no finger position. Only once you move beyond some threshold will the touchpad send a finger position. This caused a number of issues when using the touchpad.

Fixing this is relatively simple: we merely need to tell the Cypress that it isn't a clickpad and hope that doesn't cause it some existential crisis. The proper way to do this is this kernel patch here by Hans de Goede. Until that is in your kernel, you can override it with a xorg.conf snippet:

Section "InputClass"+Section "InputClass"
        Identifier "Disable clickpad for CyPS/2 Cypress Trackpad"
        MatchProduct "CyPS/2 Cypress Trackpad"
        MatchDriver "synaptics"
        Option "ClickPad" "off"
EndSection
This snippet is safe to ship as a distribution-wide quirk.

Friday, March 21, 2014

Stacking xorg.conf.d snippets

We've had xorg.conf.d snippets for quite a while now (released with X Server 1.8 in April 2010). Many people use them as a single configuration that's spread across multiple files, but they also can be merged and rely on each other. The order they are applied is the lexical sort order of the directory, so I recommend always prefixing them with a number. But even within a single snippet, you can rely on the stacking. Let me give you an example:

$ cat /usr/share/X11/xorg.conf.d/10-evdev.conf
Section "InputClass"
    Identifier "evdev touchpad catchall"
    MatchIsTouchpad "on"
    MatchDevicePath "/dev/input/event*"
    Driver "evdev"
EndSection
$ cat /usr/share/X11/xorg.conf.d/50-synaptics.conf
Section "InputClass"
    Identifier "touchpad catchall"
    MatchIsTouchpad "on"
    MatchDevicePath "/dev/input/event*"
    Driver "synaptics"
EndSection
The first one applies the evdev driver to anything that looks like a touchpad. The second one, sorted later, overwrites this setting with the synaptics driver. Now, the second file also has a couple of other options:
Section "InputClass"
    Identifier "Default clickpad buttons"
    MatchDriver "synaptics"
    Option "SoftButtonAreas" "50% 0 82% 0 0 0 0 0"
EndSection
This option says "if the device is assigned the synaptics driver, merge the SoftButtonAreas option". And we have another one, from ages ago (which isn't actually needed anymore):
Section "InputClass"
    Identifier "Disable clickpad buttons on Apple touchpads"
    MatchProduct "Apple Wireless Trackpad"
    MatchDriver "synaptics"
    Option "ClickPad" "on"
EndSection
This adds on top of the other two, provided your device has the name and is assigned the synaptics driver.

The takeaway of this is that when you have your own xorg.conf.d snippet, there is almost never a need for you to write more than a 5-line snippet merging exactly that one or two options you want. Let the system take care of the rest.

Wednesday, March 19, 2014

X.Org synaptics support for the Lenovo T440, T540, X240, Helix, Yoga, X1 Carbon

Updates: 30 April 2014, for the new INPUT_PROP_TOP_BUTTONPAD

This is a follow-up to my post from December Lenovo T440 touchpad button configuration. Except this time the support is real, or at least close to being finished. Since I am now seeing more and more hacks to get around all this I figured it's time for some info from the horse's mouth.

[update] I forgot to mention: synaptics 1.8 will have all these, the first snapshot is available here

Lenovo's newest series of laptops have a rather unusual touchpad. The trackstick does not have a set of physical buttons anymore. Instead, the top part of the touchpad serves as software-emulated buttons. In addition, the usual ClickPad-style software buttons are to be emulated on the bottom edge of the touchpad. An ASCII-art of that would look like this:

+----------------------------+
| LLLLLLLLLL MMMMM RRRRRRRRR |
|                            |
|                            |
|                            |
|                            |
|                            |
|                            |
| LLLLLLLL          RRRRRRRR |
+----------------------------+
Getting this to work required a fair bit of effort, patches to synaptics, the X server and the kernel and a fair bit of trial-and-error. Kudos for getting all this sorted goes to Hans the Goede, Benjamin Tissoires, Chandler Paul and Matthew Garrett. And in the process of fixing this we also fixed a bunch of other issues that have been plaguing clickpads for a while.

The first piece in the puzzle was to add a second software button area to the synaptics driver. Option "SecondarySoftButtonAreas" now allows a configuration in the same manner as the existing one (i.e. right and middle button). Any click in that software button area won't move the cursor, so the buttons will behave just like physical buttons. Option "HasSecondarySoftButtons" defines if that button area is to be used. Of course, we expect that button area to work out of the box, so we now ship configuration files that detect the touchpad and apply that automatically. Update 30 Apr: Originally we tried to get this done based on the PNPID or DMI matching but a better solution is the new INPUT_PROP_TOPBUTTONPAD evdev property bit. This is now applied to all these touchpads, and the synaptics driver uses this to enable the secondary software button area. This bit will be aviailable in kernel 3.15, with stable backports happening after that.

The second piece in the puzzle was to work around the touchpad firmware. The touchpads speak two protocols, RMI4 over SMBus and PS/2. Windows uses RMI4, Linux still uses PS/2. Apparently the firmware never got tested for PS/2 so the touchpad gives us bogus data for its axis ranges. A kernel fix for this is in the pipe. Update 30 Apr: every single touchpad of this generation needs a fix. They have been or are being merged.

Finally, the touchpad needed to be actually usable. So a bunch of patches that tweak the clickpad behaviours were merged in. If a finger is set down inside a software button area, finger movement does no longer affect the cursor. This stops the ever-so-slight but annoying movements when you execute a physical click on the touchpad. Also, there is a short timeout after a click to avoid cursor movement when the user just presses and releases the button. The timeout is short enough that if you do a click-and-hold for drag-and-drop, the cursor will move as expected. If a touch started outside a software button area, we can now use the whole touchpad for movement. And finally, a few fixes to avoid erroneous click events - we'd sometimes get the software button wrong if the event sequence is off.

Another change changed the behaviour of the touchpad when it is disabled through the "Synaptics Off" property. If you use syndaemon to disable the touchpad while typing, the buttons now work even when the touchpad is disabled. If you don't like touchpads at all and prefer to use the trackstick only, use Option "TouchpadOff" "1". This will disable everything but physical clicks on the touchpad.

On that note I'd also like to mention another touchpad bug that was fixed in the recent weeks: plenty of users reported synaptics having a finger stuck after suspend/resume or sometimes even after logging in. This was an elusive bug and finally tracked down to a mishandling of SYN_DROPPED events in synaptics 1.7 and libevdev. I won't provide a fix for synaptics 1.7 but we've fixed libevdev - please use synaptics 1.8 RC1 or later and libevdev 1.1 RC1 or later.

Update 30 Apr: If the INPUT_PROP_TOPBUTTONPAD is not available on your kernel, you can use DMI matching through udev rules. PNPID matching requires a new kernel patch as well, at which point you might as well rely on the INPUT_PROP_TOPBUTTONPAD property. An example for udev rules that we used in Fedora is below:

ATTR{[dmi/id]product_version}=="*T540*", ENV{ID_INPUT.tags}="top_softwarebutton_area"
and with the matching xorg.conf snippet:
Section "InputClass"
        Identifier "Lenovo T540 trackstick software button buttons"
        MatchTag "top_softwarebutton_area"
        Option "HasSecondarySoftButtons" "on"
        # If you dont have the kernel patches for your touchpad 
        # to fix the min/max ranges, you need to use absolute coordinates
        # Option "SecondarySoftButtonAreas" "3363 0 0 2280 2717 3362 0 2280"
        Option "SecondarySoftButtonAreas" "58% 0 0 8% 42% 58% 0 8%"
EndSection
Update 30 Apr: For those touchpads that already have the kernel fix to adjust the min/max range, simply specifying the buttons in % of the touchpad dimensions is sufficient. For all other touchpads, you'll need to use absolute coordinates.

Fedora users: everything is being built in rawhide Update 30 Apr:, F20 and F19. The COPR listed in an earlier version of this post is not available anymore.

Monday, March 10, 2014

Using git - the next level

There's a million tutorials out there how to learn git. This isn't one of them. I'm going to assume that you learned git a while ago, you've been using it a bit and you're generally familiar with its principles. I'm going to show is a couple of things that improved my workflow. Chances are, it will improve yours too. This isn't a tutorial though. I'm just pointing you in the direction of things, you'll have to learn how to use them yourself.

Use tig

Seriously. Don't tell me you use gitk or git log is good enough for you. Use tig. tig is to git log what mutt is to mail(1). It has been the source of the biggest efficiency increase for me. Screenshots don't do it justice because the selling point is that it is interactive. But anyway, here are some official screenshots: tig blame shows you the file and the commits, you just need to select the line, hit enter and you see the actual commit. The main view by default shows you tags, branch names, remote branch names, etc. So not only do you immediately know which branch you're on, you will see local branches that have been merged, tags that have been applied, etc. It gives you an awareness that git log doesn't. Do yourself a favour, install it, use it for a day or two and I'm pretty sure you won't go back.

tig also supports custom configurations. Here is my $HOME/.tigrc:

bind generic X !git cherry-pick -x %(commit)
bind generic C !git cherry-pick %(commit)
bind generic R !git revert %(commit)
bind generic E !git format-patch -1 %(commit)
bind generic 0 !git checkout %(commit)
bind generic 9 !git checkout %(commit)~
bind generic A !git commit --amend -s
bind generic S !git show %(commit)
So with a couple of key strokes I can cherry-pick, export patches, revert, check out a single tree, etc. Especially cherry-picking is extremely efficient: check out the target branch, run "tig master", then simply select each commit, it "C" or "X" and done.

Use branches

Anytime it takes you more than 5 minutes to fix an issue, create a new branch. I'm getting torn between multiple things all the time. I may spend a day or two on one bug, then it's back to another, unrelated issue. With the review requirements on some projects I may have multiple patches waiting for feedback, but I can't push them yet. Hence - a branch for each feature/bugfix. master is reserved for patches that can be pushed immediately.

This approach becomes particularly useful for fixes that may need some extra refacturing. You start on a feature-based branch, but halfway through realise you need a few extra patches to refactor things. Those are easy to review so you send them out to gather reviews, then cherry-pick them to master and push. Back to your feature branch, rebase and you're done - you've managed two separate streams of fixes without interference. And most importantly, you got rid of a few patches that you'd otherwise have to carry in your feature branch.

Of course, it takes a while to get used this and it takes discipline. It took me a few times before I really managed to always work like this but the general rule for me is now: if I'm hacking on the master branch, something is off. Remember: there's no real limit to how many branches you can create - just make sure you clean them up when you're done to keep things easy for your brain.

Use the branch names to help you. You can rename branches (git branch -m), so I tend to name anything that's a bigger rewrite with "wip/somefeature" whereas normal bug fixes go on branches with normal names. And because I rebase local feature branches it doesn't matter what I name them anyway, the branches are deleted once I merge them. Branches where I do care about the branch history (i.e. those I pull them into master with a merge commit) I rename before pulling to get rid of the "wip" prefix.

Use branch descriptions

Hands up if you have a "devel" branch from 4 months ago. Hands up if you still remember what the purpose of that branch was. Right, I didn't think so. git branch --edit-description fires up an editor and lets you add a description for the branch. Sometimes a single sentence is enough to refresh your memory. Most importantly: when you task-switch to a different feature, edit the description to note where you left off, what the plan was, etc. This reduces the time to get back to work. git config branch.<branchname>.description shows you the description for the matching branch.

I even have a git hook to nag me when I check out a branch without a description. Note that branch descriptions are local only, they are not pushed to the remote.

Amend and rebase until the cows come home

The general rule: what is committed, doesn't get lost. At least not easily, it is still in the git reflog. So commit when you think you're done. Then review, test, add, and git commit --amend. That typo you made in line 4 - edit and amend. I have shell aliases for amend, rbs (git rebase -i) and rbc (git rebase --continue), and almost every commit goes through at least 3 amends (usually one because I missed something, one for that typo, one for commit log message editing). Importantly: it doesn't matter how often you amend. Really. This is local only, no-one cares. The important thing is that you get to a good patch set, not that you get there with one commit.

git commit --amend only modifies the last commit, to go back and edit the past, you need to rebase. So, you need to

Learn how to rebase

Not just the normal git rebase, the tutorials cover that. Make sure you know how to use git rebase --interactive. Make sure you know how to change the ordering of a commit, how to delete commits, how to abort a rebase. Make sure you know how to squash two commits together and what the difference is between squash and fixup. I'm not going to write a tutorial on that, because you can find the documentation is easy enough to find. Simply take this as a hint that the time you spend learning how to rebase pays off. Also, you may find git squash interesting.

And remember: even if a rebase goes bad, the previous state is still in the reflog. Which brings me to:

Learn how to use the reflog

The git reflog is the list of changes in reverse chronological order of how they were applied to the repository, regardless what branch you're on. So HEAD@{0} is always "whatever we have now", HEAD@{1} is always "the repository before the last command". This doesn't just mean commits, it remembers any change. So if you switch from branch A to branch B, commit something, then switch to branch C, HEAD@{3} is A. git reflog helpfully annotates everything with the type, so you know what actually happened. So for example, if you accidentally dropped a patch during a rebase, you can look at the reflog, figure out when the rebase started. Then you either reset to that commit, or you just tig it and cherry-pick the missing commits back onto the current branch. Create yourself up with a test git repository and learn how to do exactly that now, it'll save you some time in the future.

Note that the reflog is local only. And remember, if it hasn't been committed, it's not in the reflog.

Use a git push hook

Repeat after me: echo make > .git/hooks/pre-push. And no more embarrassment for pushing patches that don't compile. I've made that mistake too many times, so now I even use my own git patch-set command that will run a hook for me when I'm generating a patch set to send to a list. You might want to make the hooks executable btw.