Skip to content


Lindy Hop is physics

This plain truth becomes obvious when we think about the flow of energy in the dance, changes in angular and linear momentum, friction between the floor and your Aris Allens, the spring constant in the connection – the list goes on and on.

This isn't a new concept. For instance, recently, I found the following great quote:

"Lindy is 50% Newtonian physics, 30% musicality and 20% magic"

I would argue, however, that this quote is wrong. Lindy hop isn't about Newtonian physics – sure, the descriptions of inertia and conservation laws can come in handy, and leads are sometimes known to jokingly discuss the concept of a perfectly spherical follow in vacuum. But fundamentally, the dance isn't about particles. It's about the superposition of particle and wave properties of the motion to create the dance.

That's right, Lindy Hop is a dance best described by quantum physics.

Conversely, quantum physics is best described by Lindy Hop, as we learned earlier this week from Krister Shalm and his Project Q.

The first step towards studying Quantum Lindy Hop, of course, is to understand its classical wave dynamics. As a data scientist who spent years playing around with waves for his Ph.D., I accept the challenge. I hereby inaugurate Project W, a.k.a. Lindy Hop Science (1). Wanna help? Send me an email!

(1) Project W for waves. Project C for classical sounded kind of lame.

Org2blog setup miscellany

The other day I had to spend some time figuring out how to do the following things in org2blog:

  • display LaTeX properly
  • embed YouTube videos
  • clear org2blog keybindings

All are very easy, but took a little bit of digging, so here it is recorded for posterity.

First, there is a plethora of LaTeX plugins for WordPress. I favor the ones that use MathJax; after all, we live in the year 2012. Converting LaTeX to images is so 1990s. For MathJax support, you can use the excellent plugin by zhiqiang. Note that the syntax for inline vs display formulas is slightly different than what one might be used to (note the exclamation marks). On the org2blog side, you have to make sure to set

(setq org2blog/wp-use-wp-latex nil)

otherwise, the LaTeX syntax generated is incompatible with the plugin.

YouTube videos can be embedded using the Artiss youtube embed plugin; in the org2blog you have to specify something like

#+BEGIN_HTML
[shortcode surrounding youtube video code, as per the Artiss docs]
#+END_HTML

Finally, the default org2blog bindings are non-standards-compliant in that they use the C-c [letter] keys, which are reserved for the user. In particular, the bindings were interfering with parts of my setup. The solution was to clear the bindings and define my own. Clearing the bindings takes the following form:

(add-hook 'org2blog/wp-mode-hook
          '(lambda () (setq minor-mode-map-alist
                       (assq-delete-all 'org2blog/wp-mode minor-mode-map-alist))))

Overall, I am very happy with org2blog. It allows me to blog right from within my preferred computing / brainstorming / writing environment. I like to think that I am not one of those Emacs users that tries to put every aspect of his or her life in some Emacs buffer somewhere, but this little lifehack is just too sensible to ignore.

Tagged with , .

Deriving the volume of an n-dimensional hypersphere in 3 minutes

What's bigger, the unit circle or the unit sphere?

This is a trick question, because L^2 and L^3 are incompatible units, so the fact that the area of one is less than the volume of the other (i.e. \pi < 4\pi/3 ) doesn't tell you much. So let's ask a different question: if we inscribe an n-sphere inside an n-cube, is a unit circle bigger or smaller, relative to its bounding square, than the unit sphere relative to its bounding cube? And in general, what will be the ratio of their volumes as a function of n ?

Suppose we fix the radius of the n-sphere to be 1. The edge length of the n-cube, then, is 2, and its volume is 2n. So, that means that the ratio of the volumes is (1/2)n, times some prefactors. Right?

Right. But the form of the prefactor here is quite fascinating. Here's what the volume of the n-sphere is:

\frac{\pi^{n/2} r^n}{\Gamma(1+n/2)}.

See that Gamma function in the denominator? It grows like (n/2)! , meaning that our original estimate of the hypersphere/hypercube volume ratio as (1/2)n is quite a few orders of magnitude off for even moderate n. As n grows, the volume of a unit n-sphere goes to zero super-exponentially.

This has an important implication when you want to cluster high-dimensional data. Intuitively, any clustering algorithm (e.g. k-means) involves drawing a boundary around a set of points that lie within a certain radius of their mean. But what we just found is that if we draw a sphere of even a relatively large radius around points in n-dimensions, for n larger than about 30, the volume of that region enclosed by the spherical boundary is approximately zero, meaning that it's highly unlikely to have any points! In that sense, everything is far apart in high dimensions. This is known as the curse of dimensionality.

Volumes of n-spheres are useful in other contexts (for instance, in statistical mechanics). In fact, back in my physics days I learned a very cool and easy derivation of the n-sphere volume formula above. I like it so much that I made not one but two videos about it. Below you can see me doing the derivation in 3 minutes flat. There is also the longer version where I do the same steps, but a little more slowly and methodically. Enjoy!

Tagged with .

Emacs + Evil = ecumenicalism

Early on in the history of the editor wars, adherents of Emacs banded together to form a religion. As a result of this ecclesiastical schism, vi came to be known as "the editor of the beast". It comes as no surprise that something denounced by a particular religious sect turns out, in reality, to be very awesome, useful, and generally good for society.

I am talking, of course, about the power of modal editing.

The ubiquitous, second-nature process of reading and writing belies the fact that working with text is a subtle, complex, and multifaceted task. Moreover, in our interactions with text we typically spend the majority of the time reading, searching, and editing. This is doubly true for structured text, such as Wikipedia markup or computer programs. Key-chord-based text editors make this process needlessly slow and inconvenient.

But this post isn't just another vi/Vim paean. This post is about Evil, the Extensible Vi Layer for Emacs written by Frank Fischer and Vegard Øye. It allows to combine the incredible power of Emacs with the ergonomics, terseness, and efficiency of Vim, and if you are an Emacs user, you should check it out. If you are a Vim user, here's your chance to switch to a better editor without having to retrain the muscle memory.

Currently, most Evil users seem to come from the Vim camp. As a result, Evil's defaults skew towards Vim behavior. This makes it somewhat difficult to adopt for us Emacs users: we have to face the double task of learning Vim, as well as figuring out how to make the modal editing paradigm work smoothly with the existing Emacs usage patterns. As someone that has done this journey, I would like to share some tips.

So let's assume you successfully installed Evil. Great! Now, what exactly did you get yourself into? Well, effectively, all Evil does is redefine the key bindings depending on the context, or "state". As a user, you will primarily deal with three states: normal (for navigating/editing), insert (for typing characters), and motion (used for read-only buffers such as help). There is also the Emacs state, which simply gives you familiar Emacs behavior, the same as what you'd get if you turned off Evil.

One option, therefore, is to use Evil's Emacs state as a default, using C-z to toggle between it and the normal state. However, I don't recommend this approach, as one would be tempted to revert to old Emacs habits, thereby missing 90% of the awesomeness Evil provides. Let's just plunge in:

(setq evil-default-state 'normal)

Now do a Google search for a good Vim tutorial, and experiment lots!

Issues you might encounter as an Emacs user

Insert state clobbers some useful Emacs keybindings

The solution to this is to clear the insert state keymap, leaving you with unadulterated Emacs behavior. You might still want to poke around the keymap (defined in evil-maps.el) and see if you want to salvage some useful insert state command by rebinding them to keys of your liking. Also, you need to bind ESC to putting you back in normal mode. So, try using this code. With it, I have no practical need to ever switch to Emacs state.

(setcdr evil-insert-state-map nil)
(define-key evil-insert-state-map
  (read-kbd-macro evil-toggle-key) 'evil-emacs-state)

I want the keybinding X to work in Evil!

You can always override or add bindings to any Evil state. Just use something like this:

(define-key evil-normal-state-map "\C-r" 'isearch-backward)
(define-key evil-normal-state-map "\C-e" 'evil-end-of-line)
(define-key evil-motion-state-map "\C-e" 'evil-end-of-line)

Evil bindings for key X shadow the default bindings in mode Y

A common culprit here is the return key, which is ordinarily bound to evil-ret (a command that, as of this writing, doesn't know about what return is supposed to do in a current mode).

A crude but effective solution is to change Evil bindings on a per-state, per-mode basis, like so:

(evil-declare-key 'motion completion-list-mode-map (kbd "<return>") 'choose-completion)
(evil-declare-key 'motion completion-list-mode-map (kbd "RET") 'choose-completion)
(evil-declare-key 'motion browse-kill-ring-mode-map (kbd "<return>") 'browse-kill-ring-insert-and-quit)
(evil-declare-key 'motion browse-kill-ring-mode-map (kbd "RET") 'browse-kill-ring-insert-and-quit)
(evil-declare-key 'motion occur-mode-map (kbd "<return>") 'occur-mode-goto-occurrence)
(evil-declare-key 'motion occur-mode-map (kbd "RET") 'occur-mode-goto-occurrence)

Note that I am using both the RET and <return> forms to make sure the key works both in terminal and under X.

This issue becomes more tricky in "read-only" modes that use letter keys for navigation (e.g. info, dired, ibuffer). It's not obvious to me what the best practices are for such modes. Should the Emacs bindings shadow Evil normal state? Does insert or normal state make more sense as the default? Currently, I don't have clear-cut answers.

I don't want Evil to ever touch keybinding X

This can too be arranged! Define the following function:

(defun evil-undefine ()
 (interactive)
 (let (evil-mode-map-alist)
   (call-interactively (key-binding (this-command-keys)))))

Now, to make sure that Evil's normal state never touches TAB, just wire this fall-through binding like so:

(define-key evil-normal-state-map (kbd "TAB") 'evil-undefine)

Mode X should start in normal state, but mode Y should start in insert state

Use evil-set-initial-state:

(evil-set-initial-state mode-x 'normal)
(evil-set-initial-state mode-y 'insert)

Doing a copy via M-w loses a character

If you are running Emacs 24, see if you are being affected by this bug.

My selection is off by a character

This can be somewhat subtle, as Emacs and Vim have different defaults regarding the visual/transient mark. In particular, in Emacs, the last selected character is always the one before the point. This makes operating on text from left to right and from right to left asymmetrical: if you wanted to select the string 123, and your cursor is on the 1, you press C-SPC,C-3,C-f. However, if your cursor is on the 3, you first have to move it past the 3, and only then do C-SPC,C=3,C-b. In Vim, on the other hand, the last selected character is always under point, so you'd just do v2l if the cursor is no the 1, or v2h if the cursor is on the 3. The fact that in Emacs, you always deal with this asymmetry, whether you are aware of it or not, can lead to selections (or, in general, cursor positioning) being off by a character when you use commands that are, conceptually, the same in Emacs and Vim, but differ in their treatment of character under point vs character before point. The good news is that Evil provides the following setting, which might help with correct character selection at beginnings and ends of lines:

(setq evil-want-visual-char-semi-exclusive t)

You might also look into changing Vim's default behavior whereby the cursor moves back one space (although this behavior makes a fair amount of sense). For this, you could look into

(setq evil-move-cursor-back nil) ;;and maybe also:
(setq evil-highlight-closing-paren-at-point-states nil) 

You could also try playing with (setq evil-visual-char 'exclusive), but I personally see no good reason to use that setting. In my own configuration, I just turned on evil-want-visual-char-semi-exclusive and it's been working well for me thus far.

Favorite Vim / Evil usage patterns

I will keep this section brief, since I am relatively new to the Vim way of editing text. Here are a few things that I have found handy or elegant in my 1.5 months with Evil. Some of these are pretty trivial, and some have Emacs equivalents (either built-in or via add-on Elisp). However, everything just feels cleaner without having to press C or M all the time.

Searching via /

We all know that isearch is the way to navigate text in Emacs buffers, and / is very similar to isearch. Because / is ergonomically superior to C-s, I find myself using the search-to-navigate paradigm much more than with my old Emacs setup.

Block selection

In Vim/Evil, C-v allows you to select rectangles and do things like insert or paste text before every line in a rectangle. In Emacs, CUA rectangle mode does something similar, but you have to enable it. In Evil, it comes built-in.

Combining search, motion, deletion, and selection commands

Such combinations are very powerful. Here are a few examples to give you a flavor of what I am talking about:

  • d/foo[RET]: deletes from point to string "foo"
  • dfa: deletes from point to character "a", inclusive
  • cta: deletes from point to character "a", exclusive, and puts you in insert mode
  • viw: selects inside word
  • vfa;: selects from point until the second occurrence of char "a", inclusive
  • yi): copy text inside parens
  • di": delete text inside double quotes

Operating on surrounding delimiters (quotes, parentheses, etc.)

These are enabled by the port of Vim's surround plugin, available here. You can do things like change double quotes to single quotes via cs"', surround words with HTML tags, and if you have selected some text, surround it with delimiters via e.g. s) or s'.

Defining your own normal mode commands

For instance, I have gj mapped to org-goto in org-mode, and gb mapped to ido-switch-buffer. I like those way more than the original bindings.

One can take this idea further and create keymaps starting with a dedicated leader key (, has been suggested as a good leader key choice). See this discussion for more insight into how one might do this via a plugin or Emacs built-in keymap functions.

Where to learn more

Evil is a relatively new project, and resources online are somewhat sparse at the moment. Some of the useful resources out there are:

  • Evil on Emacswiki
  • Official mailing list
  • Michael Markert's Evil config file is an excellent example of how one might want to customize Evil bindings and behavior, and do so in a clean manner. There are lots of goodies there, my favorite being a method to dynamically change the look of the cursor based on the state (insert vs normal). (Note that the code in that file sometimes relies on external utility functions.)
  • For a quick run-down of Evil's internals, consult the PDF or Info documentation that comes with the code.

I would have very much liked to include some Vim tutorials here, for the benefit of Emacs users, however, most of the materials that I have stumbled upon seem to be either too basic or too advanced. If you have something that you like, please leave a comment!

Conclusion

Most of the choices we make result in imperfect compromises, which makes it easy to lapse into the "grass is greener" mentality. I am a lindy hopper that envies salsa dancers, and a Linux/Android user that evangelizes all things Apple. Up until recently, I've been an Emacs devotee that coveted modal editing. Today, however, my CapsLock key is mapped to ESC instead of Ctrl, and the world is a slightly more perfect place – all thanks to the powers of Evil.

Tagged with .

Shake that grating

Using phonon scattering and temporal degreees of freedom for superresolution at the nanoscale

(This means "we can see very small things if we shake them just right".)

Have you ever wondered why we can't see viruses under a microscope? The answer to this question is the diffraction limit. But what is the diffraction limit, exactly? In signals processing terms, it is the spatial frequency bandwidth for propagating optical waves. Using optical microscopes to look at a 130 nm flu virus is akin to trying to pick out the sounds of piccolos in a Strauss concerto, over a very bad cell phone connection. You can probably discern the piccolos if you are right in the concert hall, but the task is hopeless for someone on the other end of the line. Likewise, in optics, you can see arbitrarily small things – provided you are close enough to them so that you can pick up the signal from "the piccolos" (in physics, we call them evanescent waves). That is to say, your detector must be placed in the "near field", which is the area less than a micron from the object.

This, unfortunately, is not always practical or easy to do. But over the years researchers have come up with various tricks to get around the bandwidth problem. One conceptual approach is to take the high frequency information, encode it using low-frequency data, transmit it to the far-field detector, and decode it there. This idea has been around for several decades, but the encoding/decoding step is nontrivial to accomplish, and so this scheme hasn't seen adoption in applications.

Recently, we came up with an unusual new way to perform the high spatial frequency encoding, which may turn this around. In order to lower the spatial frequency of the light wave for transmission, it is necessary to scatter its evanescent components on some periodic subwavelength structure. We showed that this is possible to do with a dynamic phonon grating, validating Jacob Khurgin's original idea.

The phonon grating creates a periodic potential that converts evanescent waves to propagating optical signals. Because it's a dynamic structure, there is much flexibility in encoding and processing the signal. Most intriguingly, phonon scattering is associated with the temporal frequency (i.e. energy) shift of the incident photons. Because of this, it is possible to uniquely detect the scattered signal in the far field using coherent detection, greatly simplifying the decoding step.

This paper was a long time in the making, and it is finally getting some attention, perhaps due to the fact that nano-opto-mechanics with phonons is becoming a popular research area. It's about time the lattice vibration quasiparticles got some good publicity, they certainly deserve it!

A related project, in which we model a system where the scattering is performed by a chirped anisotropic nanostructure (but ultra-high frequency vibrations are still used to aid coherent detection) also got published very recently. In fact, it is the first article in the first 2012 issue of APL (mendeley link). I am currently taking the corresponding issue/article designation of {100,011101} as a divine sign that I should do more programming projects. (I believe that in the coming few months the Almighty will be pleased on that angle.)

Tagged with , , .

Integrating org-mode HTML export with a WordPress theme

Org-mode exporting capabilities are excellent, but making a freshly-exported HTML project look good with your WordPress theme might seem non-trivial. As a scientist / analyst / statistician you are way overqualified to be dealing with the minutiae of PHP and CSS, and so you might have resigned yourself to less-than-stellar formatting.

Luckily, the recipe for converting an org-exported HTML file into somethign that looks like an integral part of your blog turns out to take 4 easy steps. Let's assume that I'm in the www/org/project/ directory, and the WordPress blog is in www/blog/ directory.

  1. Make a new file, e.g. project.php, and start it with

    <?php
    define('WP_USE_THEMES', false);
    require('../../blog/wp-blog-header.php');
    get_header();
    ?>
  2. Paste in the contents of the <style> and <script> tags from the org-exported HTML header
  3. Paste in the contents of the org-exported <body>
  4. Paste <?php get_footer(); ?> at the end of the file.

You are done! You are free to mess with minor details (centering of the title, location of the org-exported postamble – sometimes it makes sense to move it into the contents div), but you've achieved the main goal: your spectacular analysis looks awesome. Here's an example of the above procedure in action!

Tagged with , .

Of Russian cats and server names

I name my computers after cats. Not all of them, mind you, and I pick the names carefully. My workhorse servers are Begemot and Matroskin. Today, as I was doing some admin work, I started wondering what I would do if I had to name more machines, given that I'd rather not borrow names from T. S. Eliot.

No other fictional Russian feline can match Begemot or Matroskin in their charisma and/or raw satanic power, but for both of them I found a counterpart of sorts: Vasilii from Monday Begins on Saturday (another talking cat), and Leopold, another cartoon character.

In the process of looking for fictional Russian cats, I stumbled upon a series of Sergei Alimov's magnificent sketches for a never-produced animation feature based on Master and Margarita. This is, in fact, why I'm writing the post.

http://dnquark.com/blog/wp-content/uploads/2012/01/wpid-mm_sketch1.jpg

I am very taken with this simple charcoal rendition of Satan and his cat, black silhouettes overlooking the busy landscape of 1930's Moscow, wondering about the inner world of humans inhabiting the gray communal apartments.

The rest of the sketches are almost equally great, and convey very well the spirit of the book: at times philosophical, at times irreverent, and always just plain hilarious.

Tagged with , .

Org-mode as a blogging platform

I first noticed that technology tends to come full circle the first time I saw a CD. It was hard to miss the cyclical progression from my parents' reel-to-reel magnetic tape recorder to LPs with fairy tales about Br'er Rabbit back to (comparatively) miniature tape cassettes with '90s Soviet pop and Madonna and, finally, to a shiny disk which was just like an LP (except completely different).

The last time I mused about technology coming full circle was when I recently noticed the surging popularity of org-mode, which is essentially an add-on to a 30-year old text editing platform.

There are several reasons that Emacs/org-mode combination has such a wide appeal. Key among them is the fact that it provides a unified environment for notes, record-keeping, project-planning. The underlying format (plain text) guarantees ultimate portability, native presentation layer (in Emacs) is very simple yet very powerful, and HTML/LaTeX export functionality allows to share one's work easily.

Case in point: this post is being written in Emacs using the org2blog interface to WordPress. As of late, the amount of text in my *.org files and the amount of text on my blog have been, to put it mildly, incommensurate. Hopefully org/wordpress integration will help restore the balance to a more sensible level.

Tagged with .

Physics (v.4): a metamaterials manifesto

In studying physics, asking very simple questions often puts one at the forefront of current research and, indeed, opens whole new research areas. The field of metamaterials was developed over the last decade by posing fundamentally simple queries: can light refract negatively? What are the basic limits on the wavelength of light inside materials? What would happen if different components of the electric field vector in a light beam experienced a radically different electromagnetic environment?

As one digs further, the questions become increasingly complex and specific; not all of them will have good answers, and not all of them will even be good questions. It is easy to start getting lost and discouraged. When this happens, you can run back up the rabbit hole to the gleaming edifice of one of the basic equations. For me, this refuge is the manifestly covariant form of Maxwell's electrodynamics.

I have worked with Maxwell's equations for many years, in many different forms. Down there, in the rabbit hole, they are in their work attire, sporting unsightly divergences and curls, with constituent parameters dangling awkwardly from the field components. But here, out in the open, they are barely recognizable, dressed up as tensors and 4-vectors, conversing in classical Greek. You'd be a fool to mistake the Levi-Civita symbol for the dielectric constant. In this form, they are the equations of Einstein and Feynman, they are shining peaks in the landscape of modern physics. And yet, they are also my equations. I'm adding my strokes, however tentative and insignificant, alongside those of the greats. This gives me the inspiration to carry on.

Back down the rabbit hole I go: I'm on a mission. Optics has always attracted me by the ease with which fundamental electromagnetic and quantum mechanical abstractions become reified – often, in pretty colors – both in a lab setting, as well as in many devices that have revolutionized our world. And yet, many beautiful designs and elegant ideas are doomed to failure, due to inherent limitations of optical materials. This revolts the avowed idealist in me.

The world of applied physics is rife with trade-offs. Indeed, many are codified in the fundamental laws, such as the Heisenberg uncertainty principle. Yet most limitations are mere caprices of Nature, which tailors material parameters to its fickle, oft-inscrutable specifications. Metamaterials offer a tantalizing escape from the status quo. By custom-designing material properties, we can strike down some of the vexing compromises that limit performance and capabilities of optical devices. My mission in the rabbit hole that became my Ph.D. thesis is to eliminate the compromise that is holding back all of nanophotonics.

Many of the trade-offs in nanophotonics involve the fundamental differences between metals and dielectrics. These differences make metals appealing as short-wavelength waveguides, or emission enhancers, yet woefully unsuitable for many other applications. Dielectrics suffer from a similar fate, in reverse. Can we create a material that would behave both like a metals and a dielectric? Yes. Such materials are called hyperbolic, and they can be fabricated using modern metamaterials techniques. Some rare examples of hyperbolic materials can even be found in nature, but this is of dubious benefit to the contemporary metamaterials ideologue.

Indeed, we are past the age when our building materials were logs and mud, and we are past the age when circuit switching elements operated by thermionic emission. In the 21st century, we should get past the age when we rely on a few serendipitously found crystals to determine what we can and cannot do with photons. To be sure, future metamaterials engineers might still need to emerge from the rabbit hole to comprehend the unblemished covariant beauty of Maxwell's equations. But now, their exact from inside materials will result from our manifest destiny rather than from an accident of fate.

Tagged with , , , .

The sad state of scientific plotting software

It's unfortunate that this blog is turning into rants about scientific computing. Perhaps I am simply ethnically predisposed to doing science and kvetching.

Today's rant is brought about by my collaborators' request to make my paper figures "presentable". The reason is simple -- current figure drafts are straight up Mathematica output, relying mostly on defaults, and they suck. Let's face it -- figures produced by Mathematica can be good, but rarely great, and trying to fine-tune the appearance of frames and axes can be a daunting task.

The problem isn't that Mathematica sucks at plots -- so does most other software. The problem is that to my knowledge there are no reasonable open-source alternatives to something like Origin. So right now, if you are unsatisfied with Mathematica's plotting capabilities, your options come down to this:

- get as far as you easily can with Mathematica, then switch to Illustrator and Photoshop. Pros: guaranteed to work. Cons: time-consuming, labor-intensive, and requires Illustrator, Photoshop, and Windows.

- import data into one of {R, Matlab, Python/Matplotlib} and hope it can do what you are after. Pros: you might get the output you want. Cons: you need to know R, Matlab, or Python.

What shocks me is that in this day and age, it seems that every few years, somebody sits down to write a plotting package, and ends up reinventing the wheel. So you have a bunch of wheels, none perfectly circular; there are all sorts of bumps and protrusions, all in different places. Within R, for instance, there are at least 3 different ways to create plots (base graphics, ggplot2, lattice), and more are being created (e.g. jjplot). However, despite all the man-hours spent, we still don't have such basic things as TeX integration. In a software package that claims to be the premier graphics solution for a statistician/applied mathematician! The syntax to add formulas to plots is revolting. Even Excel can do better.

All right, rant over. I need to go generate some figures.

Tagged with , .