No Relation To Blog
Subscribe to the personal musing of Emmanuel Bernard
This entry describes how to install EncFS on Mac OS X with homebew without breaking other tools using osxfuse (like TrueCrypt).
I have been looking of a smart solution to encrypt a directory file by file. TrueCrypt is an awesome tool but does not play well with backups or Dropbox / Google Drive / SkyDrive like syncing: a TrueCrypt volume appears like a single big file which has to be synced entirely for each tiny change.
I just discovered this week-end EncFS. This is the exact tool I was looking for:
- encrypts a directory file by file: each file and its name is encrypted
- when one file is changed, only this encrypted file has to be synced
- support for each major platform (Windows, Mac OS X, Linux)
I did choose EncFS over eCryptfe for its cross platform support.
Note that some metadata still leak in such system (as opposed to TrueCrypt) but I can happily live with that.
Installing EncFS with homebrew without breaking everything else
It took me far too much experimentation for my taste. Basically EncFS in Homebrew is using fuse4x which is incompatible with osxfuse - the now preferred solution. TrueCrypt - that I still use for disk encryption - happens to be using osxfuse and my attempts to make osxfuse cohabit with fuse4x and TrueCrypt to cohabit with fuse4x failed.
Here is the working solution:
- install osxfuse via the
.dmginstall - I did install the MacFUSE compatibility layer for good measure
- install TrueCrypt (
- install the EncFS version compatible with osxfuse via brew
The last step requires to use a non standard recipe
brew install https://raw.github.com/jollyjinx/encfs.macosx/master/encfsmacosxfuse.rb
From what I gathered, osxfuse build is too complicated to be made a standard brew recipe. Too bad as I don't like to manage these externalities.
At the time, we are talking about:
- Max OS X 10.7.5
- OSXFUSE 2.5.6
- EncFS 1.7.4
Once installed, it is super easy to use
encfs ~/Dropbox/Private.enc/ ~/Private/ # or with a custom volume name encfs ~/Dropbox/Private.enc/ ~/Private/ -- -o volname="Dropbox private"
mounts the encrypted
~/Dropbox/Private.enc directory as
under the volume name
If you add or change files from
~/Private, you will see an encrypted equivalent under
To unmount, simply do
That's it. Happy secure data.
I was wrong in assuming a few things:
- Dropbox can do partial sync of a big file, so the whole file is not reuploaded in its entirety. Reference here.
- TrueCrypt does encrypt by block so changing one file does not change the whole TrueCrypt container file.
- Mac OS SpaceBundle does roughly like TrueCrypt and split a bundle into files of 8MB each. Reference here.
It remains that encfs is a nice tool but not for the reasons I mentioned :)
Thanks for Renaud and Jekyll for pointing these errors. Bad karma for Jekyll for not resisting the urge to express it like a dick.
I can't believe autocompletion is still so stupid in 2013. Alright, I said it, I feel better. It's actually not quite true and a lot of smart things are going on in Eclipse or IntelliJ IDEA but completion remains incredibly stupid when it comes to discovering an API and offering the most appropriate choice.
It is very important for fluent APIs tailored to offer only the APIs that make
sense in a particular context. So when the autocompletion offers you the full
monty of useless methods of
Object, it kills the hours of work an API designer
put into his fluent API and its discoverability. Seriously
wait() x 3!
I developed, help develop or gave advice on a number of fluent APIs in Java including:
- Hibernate Search and Hibernate Validator programmatic mapping API,
- an upcoming Hibernate OGM programmatic configuration API
- ShrinkWrap API
- Hibernate Search Query DSL API
So I feel very personally offended when something gets between my work and the end user.
I use Eclipse these days and the sort by relevance works only in so far as relevance is synonymous to alphabetical. In other words it sucks... but in a predictably alphabetical way.
IntelliJ IDEA is smarter as it puts in bold the methods hosted on the type and leave supertype methods in regular.
If you dig enough though, you can find a setting
Sort lookup items lexicographically
by opening up preferences and search of autocompletion. Weirdly enough, if you uncheck this box, IntelliJ IDEA does the better thing and put the elements in bold next to each others.
And that concludes this half rant, half tip, half call to action to IDE developers. Yes I know but it takes at least 3 halfs for a good cocktail :)
You can't reorder tokens on Google Authenticator nor edit the associated description? Read on.
I use one time passwords everywhere I can. It's a little hassle but increases security by a whole lot. And the good thing is that more and more providers offer this:
- Google Apps
- Amazon Web Services
- Your company if they are not too dinosaur-y
Go use one time passwords aka two factor authentication, you will thank me later.
I looked around on iOS and the most well known soft token application that supports both time based and event based tokens is Google Authenticator. Except that Google Authenticator's UI is really crap and buggy.
The edit button does not seem to function properly and nothing happens most of the time. The trick is to go to Legal information then back. You can now press the edit button and voilà! Things work. You can now:
- reorder the list properly
- edit the name under each token
That made my life much easier.
Google Drive is a bit confusing at first but the Dropbox like feature works reasonably well. They have a desktop client that works pretty much like the Dropbox client and syncs your local directory with the cloud.
This is quite handy when your friends have maxed out their Dropbox quota: Google Drive gives you 5 GB free. Images you store on Google Plus and files you store on Google Drive count against your quota. The Google Drive documents do not though.
Not everything is rosy in the Google Drive land though.
Trash or no trash
Data in your trash is counted as used space. It took me a while to figure that out and I could not get why my free space was lower that it should have. I don't think other services do that.
When you delete a non empty directory and then try to remove it from the trash, well nothing happens. Someone has forgotten to write the recursive part of the algorithm :) You have to go in each subdirectory and remove files and empty directories.
Simplistic client and speed
It seems to me that Dropbox syncs faster than Google Drive but I have not properly benched them so that might be subjective.
What is for sure is that the sync up cues on Google Drive are too simplistic. It tells you synchronization happens and that's it. Dropbox tells you how many files are uploaded and downloaded, what is the speed rate for each as well as the estimated time it will take. It's not gimmicky, it shows progress to the user.
Overall, I tend to stick with Dropbox despite the fact that it is twice as expensive per GB.
Hibernate documentation system uses po2xml and xml2pot to build translations. Unfortunately, Homebrew does not have a formula for it and I don't think I have the knowledge to work on such thing.
The solution is to install Macport. There is a nice UI installer. Make sure to chose the one specific to your Max OS X version.
MacPort does change your
.bash_profile. Because I want to give
Homebrew's executable priority, I make sure to put Macport
changes after homebrew in the
sudo port -v selfupdate
Then install po2xml. Unfortunately, po2xml does not come as standalone package, you have to install all of KDE
sudo port install kdesdk4
Then wait for freaking ever for everything to compile. By the way, source packaging is not eco-friendly. Think about the amount of CPU needed every time you update some package...
Once that is done, add po2xml and xml2pot to your path
And you are good to go!
I have had problems on one machine to upgrade Git from Homebrew. Let me first tell you how to fix the problem and then what homebrew is about.
The problem appeared when I tried to upgrade git
brew upgrade git
It turned out to be much more complicated than I anticipated to find the problem. The exact error message was:
Error: Failed executing: make prefix=/usr/local/Cellar/git/126.96.36.199 CC=/usr/bin/clang CFLAGS=-Os\ -w\ -pipe\ -march=native\ -Qunused-arguments\ -mmacosx-version-min=10.7 LDFLAGS=-L/usr/local/lib install (git.rb:49) These existing issues may help you: https://github.com/mxcl/homebrew/issues/8643 https://github.com/mxcl/homebrew/issues/10544 https://github.com/mxcl/homebrew/issues/11481 https://github.com/mxcl/homebrew/issues/12344 https://github.com/mxcl/homebrew/issues/12814 https://github.com/mxcl/homebrew/issues/13850 Otherwise, this may help you fix or report the issue: https://github.com/mxcl/homebrew/wiki/bug-fixing-checklist
My environment was listed as:
==> Build Environment HOMEBREW_VERSION: 0.9.2 HEAD: 53d5bfb44e8644eff1693b2a734f079d10b53043 CPU: dual-core 64-bit penryn OS X: 10.7.4-x86_64 Xcode: 4.3.3 CLT: 188.8.131.52.1.1249367152 X11: 2.6.4 @ /usr/X11 CC: /usr/bin/clang CXX: /usr/bin/clang++ => /usr/bin/clang LD: /usr/bin/clang CFLAGS: -Os -w -pipe -march=native -Qunused-arguments -mmacosx-version-min=10.7 CXXFLAGS: -Os -w -pipe -march=native -Qunused-arguments -mmacosx-version-min=10.7 CPPFLAGS: -isystem /usr/local/include LDFLAGS: -L/usr/local/lib MACOSX_DEPLOYMENT_TARGET: 10.7 MAKEFLAGS: -j2
And the last line before the error outputs were
/usr/bin/clang -isystem /usr/local/include -Os -w -pipe -march=native -Qunused-arguments -mmacosx-version-min=10.7 -I. -DUSE_ST_TIMESPEC -DNO_GETTEXT -DHAVE_DEV_TTY -DXDL_FAST_HASH -DSHA1_HEADER='<openssl/sha.h>' -DNO_MEMMEM -DSHELL_PATH='"/bin/sh"' -o git-daemon -L/usr/local/lib daemon.o libgit.a xdiff/lib.a -lz -liconv -lcrypto -lssl Undefined symbols for architecture x86_64: "_iconv_open", referenced from: Undefined symbols for architecture x86_64: "_iconv_open", referenced from: _reencode_string in libgit.a(utf8.o) _reencode_string in libgit.a(utf8.o) "_iconv", referenced from: "_iconv", referenced from: _reencode_string in libgit.a(utf8.o) _reencode_string in libgit.a(utf8.o) "_iconv_close", referenced from: "_iconv_close", referenced from: _reencode_string in libgit.a(utf8.o) _reencode_string in libgit.a(utf8.o) ld: symbol(s) not found for architecture x86_64 ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) clang: error: linker command failed with exit code 1 (use -v to see invocation) make: *** [git-daemon] Error 1 make: *** Waiting for unfinished jobs.... make: *** [git-credential-store] Error 1
It turned out that libiconv was the culprit. Simply uninstall it:
brew remove libiconv brew prune brew cleanup
brew upgrade git again and things should work now.
I found the inspiration here.
What is homebrew anyways
Homebrew is a very easy to use and maintain package manager for Mac OS X environments. Anytime you want to install one of those unix-y tools, Homebrew is your friend.
Passed the initial installation, Homebrew is as simple to use as
brew install *something*
and you are good to go. Keeping versions up-to-date are very easy too
# update brew itself brew update # update tools installed with brew brew upgrade
For example here are a few things I have installed and maintain with Homebrew (aka brew for the friends).
That also includes some Java tools:
Homebrew does not install as a privileged user - it is actually discouraged. That makes it a bit picky when permissions are not right.
I have been using this trick quite regularly with success.
ruby -e "$(curl -fsSL https://gist.github.com/raw/768518/fix_homebrew.rb)"
I think I am seriously regressing as I grow older.
I mentioned in the past moving from the popular, dynamic and UI based blog system Wordpress to the Ruby based static generator Awestruct. I also mentioned moving to Git and enjoying my command line more and more. On a tangential note, I do enjoy writing my verb in a pure textual form thanks to Markdown and use Git as my backup and memory system if you will.
I think the natural evolution for me was to move to one of those archaic editors like vi or emacs... And I tried vi... and I can't say I liked it at first.
Why oh why move to vi?
Emacs never attracted me, I have only ten fingers after all and
there is only so much flexibility in my hands.
vi has the immense benefit of literally be available everywhere
ssh in. I have always been acquainted with vi: use
i to type
ESC :q! if things go crazy.
Long time vimers keep claiming that once you get it, it's awesome. Since I have been proved wrong on Git - I thought Mercurial was better at that Git UX was an abomination for bearded people - I was rather curious.
A recent discussion with my friends of Les Cast Codeurs fame convinced me to try once again to jump on the bandwagon.
So far not so bad
In the beginning I felt that I was slow and that every line of text was draining a third of my energy for the day. And that was more than a feeling.
But it's really growing on me and this blog post is an attempt to understand why really. Yes reader, you're my private therapist.
I love good UIs but to be honest, for text documents, you don't really need it. I use MacVim with a decent set of coloring and syntax highlighting and I am happy with it. The key is that vi is extremely powerful to fine-edit text and that even if hard to learn, you feel that there is something you can gain.
What's bad about vi
For beginners or casual users, vi the most horrible piece of user experience ever built. There is no visual or otherwise cue in case you don't know how to do something, no way to navigate menus to figure what can be done. You even need to be an accomplished vi user and sort of already know what you do to begin to make sense of the help system.
Now what's interesting is that IntelliJ IDEA (my Java editor) has a vi plugin, so does Eclipse. So you can use your vi way of editing but still benefit from auto completion and code navigation.
The vi plugin for IntelliJ is not perfect. It sometimes stops working and you need to close and reopen your file, and it has the tendency to override some common shortcuts (Command click for code navigation for example). But for the most part it works. I think in 2 weeks of use, I gave up the vi mode twice. Not bad!
The other thing that's bad with vi is that there is no native project
view to navigate from one file to another in a project. I'm used to
Command-N in IntelliJ IDEA to open a Java class with a few camel case
Vi has plugins to mimic that - vi has plugins for everything by the way including ASCII diagram drawing tools. I have installed NERDTree and Command-T so far but I don't feel it's sufficient. The visual reminder of the file hierarchy is quite useful in an IDE.
Likewise, copy / pasting especially between non vi apps is not entirely fluid to me but I suspect it's a matter of training.
What made me click
There is no magic trick, you need to train your brain and your fingers to vi. But there are two things that were key to make me understand how to use vi:
- stay in insert mode (sort of the normal mode of any other editor) for the shortest period of time. Text manipulation is mostly done in command mode. So always come back to command mode.
- vi is very regular, operations are made of an action and a move. The action describes what you want to do, and the move is about what the action will influence. On top of that, you can add a multiplier to repeat the operation several times.
So when you learn a new move or a new action, you can compound them to the ones you know already and create new operations for free.
After a while you like the game. Plus it's very easy to Google a given action and find how to do it efficiently.
The arrow vs
hjkl for navigation was essentially a non problem. I use what
I am comfortable with: I started with arrows but I use almost exclusively
hjkl to move now as it makes my hand move less on the keyboard.
Likewise, I use the mouse when I feel like moving around a lot. That might
change down the road.
It has been roughly two weeks of coding and writing in vi mode and I am now feeling that when I move back to a classic editor, I don't feel as efficient as I thought I was. I'm even tempted to say that I feel a bit more efficient now in vi.
A lot of the efficiency comes from the powerful search tools. It turns out to be very efficient to navigate around text or code using small searches.
Resources and conclusion
- a stackoverflow question on the philosophy of vi commands
- the thought process of someone moving back to vi
- everything you need to know on tabulation and indenting in vi
- graphical cheat sheet
If you want to make the jump, I recommend you:
- print the cheat sheets and keep them in front of you
- read a few blogs on the philosophy of vi
- take the time to configure your (graphical) vi(m) environment
- jump and stick to it for most of your work
- use a vi plugin for your IDE: it is good to force you to use vi without losing auto completion and co you need
That's all I have, it's not a tutorial on vi, more my thoughts on the process. I'll keep you posted if I have more interesting things to say on the subject. Oh and of course this blog entry has been written in vi :)
Executive summary: I went for Instacast and so far I'm happy about it.
In its grandiose lack of thinking, Apple has decided that when syncing an iOS with iCloud it no longer copies podcast episodes from your iTunes library when you sync up.
They sort of know that a podcast is refreshed regularly so they added a
get new episode button
that brings you to the iTunes Store on your iOS. You then have to manually remember which episode
you have read and which one you want to listen to. On top of that the default is to stream them.
To download, you have to press the arrow button for long enough but not too long.
In short, horrible experience!
So I looked around to find an actual podcast application that would fill these needs:
- maintain my list of podcast subscriptions
- import my old list from the iOS Music app
- share my subscriptions via iCloud and allow for export
- download new episodes when available automatically
- can be played in the background
Two seem to be considered better than the others, Instacast and Downcast which are $2 each. I went for Instacast as it is considered the more Apple-ish one in design - I mean the good one. Downcast is more feature rich from what I have read.
So far I like it, most podcasts could be imported, though one by one by clicking the Music icon and selecting the podcasts. 3 or 4 had some URL import error but I could easily find them from their search engine. Podcasts made by yours truly are also listed. That's a decent clue of a deep catalog.
All in all, don't hesitate to switch, you won't regret it. The only sad part is that I now need two different apps to listen to my music, audiobooks and podcasts.
I got tired of using the graphical interface of VirtualBox to start and stop my VM. After a little googling, I have found a very neat script that lets you control everything from the command line.
Here is a small tip to improve your efficiency when using dynamic rebasing in Git.
I do my work on topic branches that are forked of
master. Before I push my work for review
via a GitHub pull request, I like to clean it up a bit by:
- reordering some commits
- squashing some commits together
- rewriting commit messages
Nothing fancy but it helps improve history readability.
You can of course do that by using
git rebase -i and most examples show
how you can go back in time a couple of commits.
git rebase -i HEAD^4 #go back 4 commits ago
There is a nicer and more efficient to do that when you work on topic branches
git rebase -i master
That's it. Pretty stupid but, since you can put any Git object reference, why not use the object where you started to fork off? The rebase will show you all commits between master and your branch.
If you are on a Mac, I highly recommend using GitX or one of his forks. In particular, you can amend the last commit and graphically select what should be staged and unstaged.