David Hardtke | LinkedIn

In this introductory post, I will answer Admiral James Stockdale’s famous questions:

If you told me the day I started graduate school in physics at Ohio State that I would today be managing a team of computer scientists at one of the leading social media companies on the web, I would have objected on the following grounds:

1. I hadn’t programmed a computer since an early failed attempt to learn Basic at age 10.
2. What is this “web” you speak of? And what is “social media”?
3. What does any of this have to do with experimental physics?

Of course, the day I started graduate school is the day that this chain of events started. At Ohio State, incoming graduate students typically do research with a professor during the summer before they start classes. Having recently graduated Duke with a major in Delta Kappa Epsilon and a minor in physics, I did not have clear ideas about what type of research I wanted to pursue. When I met with the department vice chair for graduate students, Richard Boyd, and he asked me with whom I wished to work for the summer. I responded with, “I don’t care, put me with somebody young.”   Professor Boyd replied that he knew just the person.

Little did I know that there was an assistant professor of Nuclear Physics, Tom Humanic, who had secretly hatched a plan with his postdoc, Sanjeev Pandey, to recruit an unsuspecting graduate student for their group. After having failed to secure a graduate student during his time at Ohio State using conventional recruiting methods, they decided to utilize bribery. Typically, a new graduate student in such a team would spend several years at the home institution before going of to the remote facility to do research. Tom and Sanjeev decided that they would send the next student to walk through their door on the boondoggle of a lifetime, an all expenses paid summer at CERN in Geneva, Switzerland. I had never heard of this research area (Relativistic Heavy Ion Physics), but I sure did like to travel. And so off I went to Europe, and it was one of the best decisions anyone ever made on my behalf – thank you professor Boyd.

It turns out that smashing heavy nuclei at very high energies does one thing very, very well – produce lots and lots of data that needs to be collected and analyzed. One thing it does not do so well is produce the nice first order phase transition we were looking for. So, while my Ph.D. thesis failed to produce the tell tale sign of the Quark Gluon Plasma, I did get very good at programming computers to apply statistical models to large scale data sets. My Ph.D. thesis from 1997 says “Experimental Nuclear Physics,” but it really should have said “Statistical Data Mining.”

I had a wonderful career as a physicist. I worked for about 5 years at Lawrence Berkeley National Lab and made some key contributions to the scientific program at the Relativistic Heavy Ion Collider. At some pint I decided that smashing heavy nuclei was no longer of personal interest and moved over to UC Berkeley and switched to Astrophysics. The experiment that I worked on at UC Berkeley, IceCube, just announced some major discoveries, and I could still be at UC Berkeley but for the femme fatale of many promising young scientists in the Bay Area: Silicon Valley.

In 2007, I left my cushy academic job at UC Berkeley to work for a two person startup in the space of Information Retrieval. Of course, I had no idea that the field was called Information Retrieval. It took me six months on the job to realize that very good people had tried to do what we were doing and had even coined a name for the technique: Implicit Revelance Feedback. Of course, Silicon Valley is all about consciously ignoring the fact that your idea probably isn’t new and that others had probably failed trying to do the same thing.

Anyways, it turns out that my training prepared me very well for work in the nascent field that has come to be known as data science. Physicists are good at learning just enough about many, many things in order to be successful. I knew just enough statistics, programming, machine learning, and large scale data mining to be very good at physics. Those are exactly the things I need to know to do job recommendations for millions of LinkedIn members. Some other things I learned as physicist that come in handy are using data to tell a story and the ability to finish large, complicated projects that involve many, many participants. Building and deploying a job recommendation system is trivial compared to getting a paper published with 500 coauthors (all of whom have opinions about where you made a mistake).

To an outsider it appears as if I am not utilzing my Ph.D., but in reality the skills and techniques I learned then are exactly what I utilize on a daily basis. And rather than applying those skills and techniques to esoteric research topics appreciated by only a few researchers in the same field, I get to change the lives (hopefully for the better) of millions of people.