When I started doing optical observing in my postdoc, I was unpleasantly surprised at how difficult it was to learn to reduce the data. Most optical astronomers use a venerable package called IRAF, which may charitably be called “user antagonistic.” There is a Python wrapper, PyRAF, which mutes some of the annoyances but is no […]
Recently, the American Astronomical Society’s Committee on the Status of Women in Astronomy (CSWA) released a report on the demographics of US astronomers throughout the academic career cycle: graduate students, postdocs, and the various ranks of professors. The major goal of the report (written by my friend, Prof. Meredith Hughes) was to assess the progress […]
At this week’s meeting of the High Energy Astrophysics Division of the American Astronomical Society, I learned that one of the teams analyzing data from a NASA observatory had run into trouble with their machine learning classifier. Their problem illustrates one of the particular challenges of machine learning in astronomy: sample selection bias. At first […]
A recent major proposal deadline gave me a chance to brush up on my LaTeX skills. As a rule, it’s better to make your proposal more concise than to play formatting tricks to squeeze more text in. For this proposal, though, I needed the big guns–for some sections the instructions alone were a significant fraction […]
Python is often the language of choice for today’s cutting-edge astronomical software. Scientists wishing to take advantage of this powerful and growing ecosystem face the hurdle of learning a new programming language. Thankfully, with the rapid growth of scientific Python, a number of excellent comprehensive tutorials have been developed, many particularly for astronomers: A CfA […]
Jessica Kirkpatrick wrote a great post describing her move from astronomical research to a data science job at Yammer. (We were classmates in grad school.) She discusses the technical skills she needed to learn (IDL alone won’t get you a tech job) as well as the differences between business and academic culture. (Peter Fiske’s book Put […]
Scientific graphics are one of the most important means we have of communicating complicated quantitative information. Here are a few ideas for improving the effectiveness of your figures: Learn a new feature of your graphics package. Most of us only use a fraction of the capabilities of our graphics programs. There’s much to be gained […]
As a scientist, most of my work consists of text files: source code, data, and papers. Keeping track of changes to these files with version control helps me avoid losing work, track down bugs, reuse code, and keep records of my research. Rather than renaming files code.py.oldversion, code.py.aug01, code.py.brokendontuse, version control gives me a single, […]
All human activities are subject to limits. Economic or technological trends determine the futures of jobs and businesses. Weather shapes agricultural yields. In athletics, the strength of an opponent’s or of one’s own body are decisive. Creativity, inspiration, and influence play roles in art and literature. Political progress requires coalition-building and persuasion. Some of these […]
The public data challenge has emerged as one response to the need for sophisticated data analysis in many sectors. The prototypical example of these competitions is the Netflix Prize, which awarded $1M for improved predictions of user movie ratings1. Kaggle provides a platform for organizations to sponsor their own challenges. The most high-profile is currently […]