Parsing Gem Versions With Awk
I installed latest Ruby (2.0.0-p0) today and along with it the smallest gem versions from the previous Ruby (2.0.0-rc1). Here is awk one-liner that helped me do that.
chruby 2.0.0-rc1
gem list | \
awk '{ gsub(/\(|\)/, ""); line = line " -v=" $NF " " $1; } \
END {print line; }' > gems-versions
chruby ruby-2.0.0-p0
gem install `cat gems-versions`
First, I switched to ruby-2.0.0-rc1 with 'chruby' in order to make a list of available gems. Then, in order to parse the list of gems I piped it to the awk command. Here is what it does:
- Removes parenthesis that enclose gem versions in each line with gsub(/\(|\)/, "").
- Concatenates gem version with its name in a single line in the form that the gem install command understands (i.e -v=<gem-version> <gem-name>) with line=line " -v="$NF" " $1. $NF and $1 denote the gem's version number and name respectively.
- Prints the concatenated line in the 'gems-versions' file at the end.
After I created the list of gems I switched to ruby-2.0.0-p0 and installed gems with: gem install `cat gems-versions`.
It is easy to adjust this to the similar use cases. If we want, for instance, to install all gem versions that are already present in the previous Ruby version the awk that will parse all gem versions will look something like this:
chruby ruby-2.0.0-p0
gem list | \
awk '{ gsub(/\)/,""); gsub(/\(|,\s/, " " $1":"); \
$1=""; line = line " " $0} END { print line }' > gems-versions
chruby ruby-2.0.0-p0
gem install `cat gems-versions`
Everything is almost the same as in the previous case except there are a couple of extra awk commands to add.
- gsub(/)/,""); gsub(/(|,\s/, " " $1":") replaces string xgem (2.0, 1.0) with xgem xgem:2.0 xgem:1.0
- $1="" sets the first field (a gem name) to the empty string so the output string becomes gem_x:1.0 gem_x:2.0 which should be valid input for the gem install command.
- line = line " " $0 concatenates the output string $0 to the line variable which will be saved to the gems-versions file.
Automount With Udisks in Arch Linux
When it became obvious that systemd was going to replace sysvinit and init scripts I switched to systemd and experienced several issues in the process. One of those was that the system couldn't automount usb drives any more.
Before the transition I used devmon which was started in .xinitrc before windows manager with ck-launch-session bash -c "devmon & ..." command and it worked seamlessly. However, after the upgrade the devmon was broken. No matter whether it was issued manually in terminal or from the .xinitrc script, it reported the following error:
Poll for media failed: Not Authorized
device: [/dev/sdb1]
...
mountdev /dev/sdb1 DNC vfat
devmon: mount /dev/sdb1 --mount-options noexec,nosuid,noatime (DNC)
Mount failed: Not Authorized
devmon: error mounting /dev/sdb1 (0)
...
udisks functions are not authorized through policykit,
so devmon cannot automount drives.
...
I also tried udiskie, another automatic disk mounter that uses udisks, and it failed to mount disk with the similar message:
attempting to mount device /org/freedesktop/UDisks/devices/sdb1 (vfat:[])
failed to mount device /org/freedesktop/UDisks/devices/sdb1:
org.freedesktop.UDisks.Error.PermissionDenied: Not Authorized
Obviously the problem was not devmon, nor udiskie. Somehow udisks functions were no longer authorized through policykit... It was clear that I should famialirize myself with polkit and find out how to authorize udisks actions.
From polkit(8) manpages I learned that in order to authorize disk mounting I should write polkit rule. These rules are placed in /etc/polkit-1/rules.d/ and /usr/share/polkit-1/rules.d/ directories in files with .rules extension. Both directories are monitored by polkitd process. If rule is changed, added or removed all rules are processed again.
So I added /etc/polkit-1/rules.d/10-udisks-automount.rules file with the following rule:
polkit.addRule(function(action, subject) {
if (action.id == "org.freedesktop.udisks.filesystem-mount" &&
subject.isInGroup("storage")) {
return polkit.Result.YES;
}
});
Basically it says that udisks mount action should be allowed to the users in the storage group. Values for action.id as well as other information about polkit authorization policies can be found in /usr/share/polkit-1/actions directory. Particularly, the policy for the udisks is placed in org.freedesktop.udisks.policy file. All rules are written in JavaScript.
After I'd added this rule, devmon and udiskie started to work as expected so I haven't searched further. For that reason I suppose that what I've presented here is probably quick and dirty solution. It should serve just as a remainder to me in the future and it might also help someone with the similar problem. More comprehensive coverage of udev, udisks and polkit can be found on the Arch wiki pages for udev and with man polkit(8).
Browse and change code history with Git
Sometimes, if I really want to understand a peace of software, I embark on browsing its whole history available in Git. I usually checkout its initial commit and than slowly make my way through the forest of subsequent commits and branches. If the project is the important one and has live and vibrant community I also consult mailing list archives for the point of time when a particular commit is made. This way, not only that I learn what were the original driving ideas of the current design, but often I gain deep insights about evolution and dynamics of the project .
I'll write now about some Git commands from its time-traveling-and-history-changing toolbox that I use in this setting.
Let's assume the following history exists and the current branch is "develop":
A---B---C---D---E---X develop
Now let's say that I'm on commit B where some particularly complicated algorithm is introduced. As I have hard time understanding the algorithm I write lots of printf's, comments and proof-of-concept code all over the place. Also, I often try to do the same thing that is done in the code differently and sometimes (especially when I make a mistake) I see why the author had chosen the particular way of doing it.
This meta-code helps me to really understand the original source and forces me to learn and adopt good practices and ideas from it. It has a tremendous value for me because if I need to check the algorithm at the point B again I will, hopefully, understand it and know how to reuse it much more quickly with this meta-code explanations. As I want to preserve this for the future I create commits along the way and usually group them in separate 'code-review' branch.
git checkout -b code-review <B-sha1>
Next, I write explanatory comments, log messages and tests for commit B in commits F, G, H on the code-review branch so my git repository looks something like this
F---G---H code-review
/
A---B---C---D---E---X develop
When I'm done with reviewing commit B I move further down the main developing line to commits C, D, and E. Let's say the code there consists just of smaller tweaks of the algorithm that is introduced at the point B. I understand everything without need to write additional meta-code and I want to integrate them in my code-review branch so I can move on to another commits down the develop branch.
The easiest way to integrate C, D and E with all previous code-review commits (F, G, H) is to use git rebase command (at this point I would be in 'code-review' branch):
git rebase <E-sha1>
So I end up with the following repository:
F---G---H code-review
/
A---B---C---D---E---X develop
However, let's say I want to integrate only commits F and G with the C, D, and E from the develop branch. In that case I would use git rebase with --onto option as follows:
git rebase --onto <E-sha1> <B-sha1> <G-sha1>
git checkout -b temp
git checkout code-review
git reset --hard temp
git branch -d temp
The first command basically says move all commits between B (not including it) and G (including it) onto E commit. After rebase is successfully finished the repository should look like this:
F---G---H code-review
/
/ F---G (no-branch)
/ /
A---B---C---D---E---X develop
There is a new (no-branch) HEAD that is in so called detached state. In other words it's simply a commit hash which isn't pointed to by a tag or a branch. Basically, whenever you check out a non-referenced head, you end up with a detached head. In practice some conflicts may arise at this stage. To efficiently resolve them I recommend git-rm-conflicts script written by John Wiegley.
Next I create 'temp' branch to point to detached HEAD, and then checkout to 'code-review'. The 'code-review' branch pointer is then forcibly moved with git reset --hard temp to point to the same place as 'temp'. At the end 'temp' branch is deleted so final repository should look like this:
F---G code-review
/
A---B---C---D---E---X develop
And for the end, here is the script that is very handy for reviewing branches and project history:
git log <B-sha1>^..HEAD --graph --decorate --pretty=oneline \
--abbrev-commit develop code-review some-other-branch
You can add -p to see detailed diffs, leave off --pretty=... to see whole log messages, or tweak it some other way to suit your needs.