Oh how I longed for that feature: Whenever I log into XFCE, I want my ssh key to be automatically added to my authentication agent. Although it took some time, I finally puzzled together all the pieces to make it work. So here it is. 3 simple steps to get auto-adding to work in Xubuntu karmic koala:
  1. Enable "start gnome environment" in XFCE settings
  2. Comment out the line "use-ssh-agent" in /etc/X11/Xsession.options. This prevents the ssh-agent from starting. Instead gnome-keyring-daemon's internal agent will be used. However ssh-agent would overwrite it's socket variable, so we disable it here.
  3. Place the following into your ~/.xprofile: if [ -n "$GNOME_KEYRING_PID" ]; then eval `gnome-keyring-daemon --start` export SSH_AUTH_SOCK fi
There you go. If it does not work after a re-login, leave a comment below. Further information regarding the GNOME startup process and the way gnome-keyring-daemon gets started (it's actually started twice, huh!) can be found in this bug report.

16. April 2010: Update for Lucid Lynx

Apparently my fix from above prevents things from working Lucid. Therefore, if you have problems, unset everything and it should work. Nice. It seems to be totally broken in lucid. After wasting already to much time for this, I filed a bug report about it, we'll see what happens: https://bugs.launchpad.net/ubuntu/+source/xfce4-session/+bug/564831.