macOS powerline and Python interpreters

2018-07-18  |   |  tool   Mac OS X  

tl;dr; using powerline and having it fail after a homebrew update? Read on.

If you have tuned a bit your terminal in macOS, you might be using Powerline. It prettities your command line enough to warrant some trouble installing it. And it does it to vim status lines too.

Now if you also use homebrew, every so often you see something like this when starting a new terminal window

-bash: /usr/local/bin/powerline-daemon: /usr/local/opt/python/bin/python3.6: bad interpreter: No such file or directory

I always fiddle with things until I get it fixed again. So here it is reported for posterity. The Python interpreter has been updated behind your back and the installed modules end up referencing the old interpreter.

For powerline, here are the steps

pip uninstall -y powerline-status
brew update && brew upgrade python
pip install powerline-status

Note that I use Python 2 and not Python 3. From what I remember, I had troubles making it run properly on v3.

Here is a more generic solution, courtesy of bnaecker at StackOverflow.

pip list | cut -d " " -f 1 > package-list.txt # Store package names without versions
pip uninstall -y $(cat package-list.txt) # Cannot use redirection
brew update && brew upgrade python
pip install $(cat package-list.txt)

Back to work.

tig and add interactive

2017-08-02  |   |  tool   git  

Tig is a textual GUI for git which lets you manipulate your git repo like GitX or SourceTree but without leaving your terminal. In particular, you can "graphically" do git add --interactive. I find it easier to do my chunk surgery when I only want to apply parts of a file change.

The documentation is a bit terse so let me give you a rundown.

Tig add interactive in action

Navigate diffs

launch tig
s # to see the status view
arrows or j/k # to select the file diff to look at
enter # to see changes in that file, this is the stage view, alternatively hit c
j/k # to navigate up and down the file lines

Select specific chunks

(go down to a specific chunk)
u # to select the whole chunk
! # to delete/reset the whole chunk change (you must be on the chunk header line) ; caution this change will be gone forever

Select specific lines

(in a chunk on a specific line)
1 (one) to add the specific line to stage
! and then y to delete/reset the specific line ; caution this line will be gone forever

To undo the staging and remove the line, select the file in the changes to be committed section
1 (one) to move the line back to the non-stage area

Other goodies

Shift+R to refresh the diff changes in case you update the file in your editor and want a updated diff

You can delete/reset changes to a whole file by hitting ! and then y in the status view. Make sure to select the right file as the changes will be gone forever.

Simplifying your double (calendar) life

2017-07-21  |   |  productivity  

If like me you live a double life (pro and personal) and like me you want to share your availability via calendars, you probably have created clones of the same event into multiple calendars. There is a better way.

Double life

I live double calendar lives.

A shared calendar tells where I am (travel, conferences, face to face meetings, holidays etc) to the team I work with. Except that I am in more than one team, so there are several of these shared calendars. Likewise, my spouse likes to know when I am out of town (or rather does not like to know). And I end up creating one event in my main calendar and one event per shared calendar.

There are many occasions where an event is duplicated for one reason or another.

Invite instead of duplicate

Max shared a pretty useful tip with me the other day. Instead of duplicating an event in each calendar, you can invite shared calendars to a single event.

Google assigns email addresses to each shared calendar. You can find this email by selecting Calendar settings on the right menu of a given calendar (down arrow). Under Calendar address, you will find an email that looks like this Also make sure that the calendar setting Auto-accept invitations is set to Automatically add all invitations to this calendar. After all it will need to accept invites to your parties :)

Add to your contacts an entry with the email address (it's easier to remember than this opaque email address). And next time you need to put an event into multiple calendars, simply invite this/these contact(s) to the event. And voilà!

MacBook Pro and the SMC cure

2017-07-11  |   |  apple   Mac OS X  

The physical parts of your MacBook (fans, ports etc) behaves erratically? I might have a cure.

A story of crashes

My MacBook Pro was getting on my nerves lately. When plugged to my Thunderbolt dock and thus my external monitor, my laptop would crash randomly. Sometimes, once every 2 weeks, sometimes several times a day. I tried many things, like not starting all of the utilities one usually uses to verify that they were not involved but I could not find a culprit.

Then one day, I saw that one of my Thunderbolt port would not accept my Thunderbolt-Ethernet adaptor while the other did (the connection did not show up). I was convinced my laptop had a faulty hardware and needed repair.

System Management Controller to the rescue

So I called Apple and the support person told me to try something first: reset the SMC (System Management Controller) by pressing Shift+Control+Option on the left side and the power button for 10 seconds while starting the computer (after a full shutdown) - full procedure here.

And voila, all my problems are gone.

What is SMC anyways?

It turns out that there is a chip that manages a lot of hardware inside your laptop: fans, LEDs, IO ports, external displays, battery, etc. So slapping that chip on the face (figuratively) might make a lot of things go better.

Using vim and Evernote

2017-03-28  |   |  tool  

I like Evernote because all my notes are in a single place and I can access it from everywhere (desktop, phone). I use it very much as my GTD reference material. But the editor is subpar compared to vim though - what isn't?


I found a way to edit my Evernote notes in vim. I use Geeknote and specifically this fork.

git clone
python2 build
pip2 install --upgrade .

(I forgot to do the last line and it created a lot of pains. I also tried the brew formulae but did not work for me.)

You need to log in and select vim as your editor

geeknote login
geeknote settings --editor vim

And from there, you're good to go to edit notes

geeknote edit --note "Moon project meeting"

And boom. I personally can now edit my notes in Asciidoc markup and be happy :)


There is a more advanced integration with vim: vim-geeknote. I am not sure I like it but I'm exploring it. It gives a navigation panel inside vim.

Make sure to set it to use the plain format and not the markdown one. Otherwise, all your notes will look weird back in Evernote.

" in your vim.rc
" Geeknote options
let g:GeeknoteFormat="plain"
" filter the relevent notebooks
let g:GeeknoteNotebookFilters=[ "my notebook" ]
" do syntax highlighting with asciidoc syntax
autocmd FileType geeknote set syntax=asciidoc

Perfect solution / future

I initially thought about keeping all my notes in sync in a directory with each note exported as Asciidoc, but I found geeknote before going in development tunnel.

I don't think I really need the directory sync but I would like to be able to have the note as a file that I can render via Asciidoctor (e.g. asciidoctor.js).

Embedding Vim options in Markdown

2017-03-06  |   |  tool  

I've been wrestling with an issue for quite a while: embed vim options into a Markdown file. I finally found a solution.


You can customize the options set when opening a specific file in Vim. This is pretty handy when you want to customize things like tab vs spaces or the language to spellcheck with.

This is typically done by adding some vim incantation as a comment in the file.

// vim: set softtabstop=2 shiftwidth=2 expandtab:
public class JavaClassExample {
  public String name;

Vim options in Markdown

I've been really struggling to get this working on Markdown because unfortunately Markdown does not have a comment syntax. That's until I remembered that Markdown defaults to HTML the minutes it does not know how to express things. <!-- --> to the rescue! Without further due, an example:

# Un exemple en Markdown

<!-- ask vim to use French to spellcheck -->
<!-- vim:set spelllang=fr : -->

<!-- ask vim to use 2 spaces for indent
     (a blog does not have lots of space) -->
<!-- vim: set softtabstop=2 shiftwidth=2 expandtab: -->

Ceci est un test montrant la validation orthographique en Français.

## Et ça marche ?

Carrément !

    public class JavaClassExample {
      public String name;


Start IntelliJ IDEA from the command line

2017-02-27  |   |  java   ide   tool  

You can start IntelliJ IDEA from the command line which is handy when you live in a terminal like me. But you need to enable that feature.

Open IntelliJ IDEA, go to Tools->Create Command-Line Launcher... and optionally adjust the location and name of the script that will start IntelliJ IDEA. Voilà! Now from your command line, you can type:

  • idea . to open the project in the current directory
  • idea pom.xml to import the Maven project
  • idea diff <left> <right> to launch the diff tool.

The generated script has an annoying flaw though, it does reference your preference and cache directories in a hard coded fashion. And for some reason the IntelliJ folks embed the version number in these directories (e.g. IdeaIC2016.2) That's annoying as it will likely break the minute you move to another (major?) version.

Antonio has a solution for that which is a simpler and more forgiving script in good anti-fragile fashion. The script is not generic and only runs for macOS.


# check for where the latest version of IDEA is installed
IDEA=`ls -1d /Applications/IntelliJ\ * | tail -n1`

# were we given a directory?
if [ -d "$1" ]; then
#  echo "checking for things in the working dir given"
  wd=`ls -1d "$1" | head -n1`

# were we given a file?
if [ -f "$1" ]; then
#  echo "opening '$1'"
  open -a "$IDEA" "$1"
    # let's check for stuff in our working directory.
    pushd $wd > /dev/null

    # does our working dir have an .idea directory?
    if [ -d ".idea" ]; then
#      echo "opening via the .idea dir"
      open -a "$IDEA" .

    # is there an IDEA project file?
    elif [ -f *.ipr ]; then
#      echo "opening via the project file"
      open -a "$IDEA" `ls -1d *.ipr | head -n1`

    # Is there a pom.xml?
    elif [ -f pom.xml ]; then
#      echo "importing from pom"
      open -a "$IDEA" "pom.xml"

    # can't do anything smart; just open IDEA
#      echo 'cbf'
      open "$IDEA"

    popd > /dev/null

The GitHub gist version of this script. It does not offer the call to IDEA's diff though. I'm from an era where we did resolve > based diff conflicts in Notepad so that does not bother me much.

I think I'll go for Antonio's solution, that will avoid some nasty WTF moments when the preference directory moves and I will have forgotten all of this.

Retraite, baby boomers et pyramide des ages

2017-01-18  |   |  economy   francais  

Je n'avais jamais bien compris pourquoi les baby boomers étaient un tel problème pour les retraites des plus jeunes. Mon hypothèse était que la retraite des baby boomers est payée par des actifs dans un système à répartition comme en France. Donc le moment difficile c'est le moment de vie active où la pression de paiement est la plus forte. Mais ces baby boomers, ils vont bien finir par mourir, et donc le système reviendra à la normale. Ce raisonnement est correct mais il y a plusieurs facteurs aggravants.

Premièrement, le phénomène de baby boom est plus long que ce que je pensais. Il correspond aux naissances de 1945 à 1975 et à un surplus de naissance de ~ 20-25% par an par rapport à la moyenne des naissances post 1975 (à la louche). Donc c'est un gros morceau à la fois dans le temps et dans le volume supplémentaire par an. Les baby boomers ont commencé à entrer en retraite il y à 5 ans et cela va se poursuivre quasi jusqu'à ce que moi je rentre en retraite (je ne suis pas né au meilleur moment apparemment).

Pyramide des ages et baby boomers

Deuxièmement, ce phénomène n'est pas compensé (ni aggravé d'ailleurs) par les naissances et l'immigration. Pour faire simple, les 1,95 enfants par femme et la légère immigration que l'on a devrait suffire à garder les tranches d'age stables. Cela donne une base rectangulaire à la pyramide des ages. Cela dit si on faisait un peu plus de bébés ou si on accueillait un peu plus d'immigration, ma retraite se porterait mieux, CQFD.

Et évidemment, les gens meurent plus tard. Merci aux progrès de médecine et de nutrition.

Donc on se retrouvera avec une pyramide en forme de gros rectangle avec un petit chapeau au dessus (appelé cylindre pour une raison qui m'échappe).

La pyramide des ages du futur

La part des personnes agées dans la population va passer de 20% à 30-32% en 2035 pour se stabiliser ensuite d'après l'INSEE. Elle devrait baisser légèrement une fois les baby boomers tous partis remplir la pyramide inversée du ciel (c'est beau non ?). Le chapeau devrait se réduire un peu en largeur donc. Mais ça c'est pour après 2060.

Vu que je suis né juste après le baby boom, j'ai globalement tout perdu (je paye et je ne serai pas payé). Sauf si la France se met à faire beaucoup de bébés (l'immigration n'étant pas très tendance en ce moment). Il fait froid, lancez vous !

Course on inverted index

2017-01-17  |   |  computer science   conference  

I gave a three hours course on inverted index to students from Telecom SudParis an engineering school here in... Paris :) It was fun to refresh my knowledge on all the fundamental structures that make Lucene what it is.

I covered quite some ground for this three hours course (a bit to much to be honest). Amongst other things: b-tree, inverted index, how analyzers and filters do most of the magic (synonym, n-gram, phonetic approximation, stemming, etc.), how fuzzy search work in Lucene (state machine based), scoring, log-structured merge and the actual physical representation of a Lucene index and a few of the tricks the Lucene developers came up with. My list of reference link is pretty rich too.

Without further ado, here is the presentation. I tend to be sparse on my slides so make sure to press s to see the speaker notes. The presentation is released under Creative Commons and sources are on GitHub.

It is a first revision and can definitely benefit from a few improvements but there is only so much time per day :)

Log-Structured Merge Tree with level-based compaction

2017-01-10  |   |  computer science  

It is surprisingly hard to find a good explanation to level-based compaction of a Log-Structured Merge Tree. It turns out that it is best explained in LevelDB's documentation. You can find the (html) details here.

This blog post is a collection of key concepts I did not grasp initially. Sort of a mental note for me. It should bring you nicely from standard size-based compaction LSM to level-based compaction.

Levelled LSM structures are useful as they limit greatly the number of files to access when reading a given key. nbrOfLevel0Files + (n-1) where n is the number of levels.

You still have levels but level 1 and above have different behaviors:

  • there is the in memory level + append only log (non ordered)
  • there is level 0 which behaves like a normal LSM level (each file is ordered but has overlapping keys)
  • level 1 and above have files containing non overlapping keys

I call segment, a given file at a LSM level (containing ordered keys), it is called sstable usually. Why do I call it segment? I come from Lucene and I have read Dune and know of sandworms. Plus segment is a much nicer word than sstable :)

The non-overlapping ranges for a given level L are not fixed in stone and are recomputed each time a compaction from level L to level L+1 occurs. Big ahah moment for me.

When a level L is merged into level L+1 (for L >= 1), one segment of L and all overlapping segments of L+1 are read. New segments are created at level L+1 from this data and a new level L+1 is created from these new segments and the existing non-overlapping ones. When compaction is done, the manifest (reference of segments) is updated and old segments are deleted. From that data, new segments at a given level are created based on:

  • size (e.g. every 2 MB)
  • as soon as the key range of a given segment overlaps with more than 10 segments at L+2

Tombstones are kept around until the last level (to make sure we hide the possibly older values in higher levels). They claim that they remove tombstones for a given key if no higher level has a segment with whose range overlaps the current key but that looks like a minor optimisation.

In LevelDB, the max size of a level is 10^L MB (e.g. 10 MB for 1, 100MB for 2 etc). Levels do increase in size exponentially though each segment is of fixed size (at least not exploding).

All this compaction only involves sequential reads and sequential writes (when done right).

I'm well aware that many improvements have been built atop this initial approach but they all rely on you understanding this first cornerstone improvement :)

Cool stuff.

Name: Emmanuel Bernard
Bio tags: French, Open Source actor, Hibernate, (No)SQL, JCP, JBoss, Snowboard, Economy
Employer: JBoss by Red Hat
Resume: LinkedIn
Team blog:
Personal blog: No relation to
Microblog: Twitter, Google+
Geoloc: Paris, France