How an AI-powered telescope is helping astronomers sift through the stars

Imagine you had as many files as stars in the sky. Some days, maybe that doesn’t feel so hypothetical. Now imagine that those files were about to explode. You can’t hope to save them all—but if you work fast, you can save as much information as possible before they disappear. 

In searching for new supernovae, astronomers face that race against the clock all the time. 

“If something has exploded, every day you wait to interrogate that thing further, you lose information because it's constantly evolving,” explains Adam Miller, Assistant Professor of Physics and Astronomy and Director for the LSSTC Data Science Fellowship Program at Northwestern University. 

That’s one reason Miller and his research assistant, Astronomy PhD student Nabeel Rehemtulla, are excited about the potential for machine learning to revolutionize the way they work. 

Over the past few years, they’ve been building the Bright Transient Survey Bot. The BTSbot uses images collected by a robotic telescope that searches for exploding stars. From there, Rehemtulla’s machine learning algorithm confirms a discovery by comparing the data against over a million historical images. In early October, the BTSbot became the first ML-assisted tool to detect, identify, and classify a new supernova autonomously.

“Many people have talked about it as a possibility,” says Miller, “But until just a few weeks ago, no one had ever actually done this.”

In building the BTSbot, Miller and Rehemtulla have not only automated the most tedious task in their workflow, they’ve given themselves and future researchers the freedom to focus on work that’s more deserving of human attention. We spoke with them to find out how they achieved this feat and what we can learn from their approach to automating their workflow.

What inspired you to build the BTSbot?
Miller: We were hoping to obtain a spectrum of everything that reached this critical brightness threshold, which is what informs the bright transient survey. The way we’re able to ensure that we get a spectrum of every single thing that’s bright is by having someone visually look at every reasonable candidate, then decide, “Yes, we will try to obtain a spectrum” or “No, we won't.” 

That decision comes with some cost because we can only take so many spectra per night. So it matters that you make that decision well. The inspiration came from the fact that I was a little bit sick of having to look at the data and do this highly repetitive task every day. I thought—couldn't we build a machine learning model that would somewhat replicate the decision-making process ?

What makes the BTSbot different from other telescopes using ML to search for supernovae?
Miller: A lot of the machine learning models that people build while looking at things like explosions are working on an extracted data space. Essentially, they’re measuring features from the images, then building models on the features as opposed to the images themselves. We said, let's try to answer this problem with the images. But the inspiration was wanting to reduce the cognitive load on myself and the other people looking at all of this data.

Nabeel, what drew you to this project?
Rehemtulla: I enjoyed the idea of bringing novel, ground-breaking technologies to time-domain astronomy [Editor’s note: this is the study of how astronomical objects change with time]. In my previous work, I brought really smart, data-driven tools to a really old method of doing this—dark matter modeling. I think the paper that originally introduced that method was 1919. We used recent smart curve fits to update and then breathe new life into that old method. 

Sorry, did you say 1919? 
Rehemtulla: Yeah, it's conceptually super simple, but you can do amazing stuff with it if you have great observational data, [and] also smart data-driven methods. 

How does the BTSbot use machine learning?
Rehemtulla: With this specific task that’s repetitive to do manually, the idea is to plug a machine learning model into that step of our workflow. I think this is the more correct way of applying ML to science. The alternative is starting with ML then looking around to find somewhere to put it. I think it makes more sense to start with a problem and see where that ML fits in that gap. 

Now that we actually have it running in production, we can sometimes see the human scanners that are still doing this manual inspection behave in some weird ways. So we can tune the model to kind of bend around their quirks, then interface better with the humans. As we've gotten into working in production, I've been babysitting it every morning, making sure it does the right things. But the idea is to be more hands off to really make use of the technology we've made.

How does your team at Northwestern collaborate with other researchers?
Miller: The overall umbrella under which we're all sitting right now is the Zwicky Transient Facility project. This is an international collaboration between researchers at more than a dozen institutions. We're all working with different and overlapping interests with the data that comes from that project.

There's one sub team that's very interested in stars in the Milky Way. The sub team we work on is interested in trying to classify every single thing that explodes outside the Milky Way—what we call extragalactic transients. There's a weekly video conference where we all jump on Zoom. We catch people up on what's been happening with the telescope, then discuss strategy.

Within that, individual researchers have a lot of freedom to say, “These are the questions that I'm most interested in.” I became interested in the question, can we automate this procedure that's taking a lot of time? I started to believe we had enough data that we could actually train a machine learning model. Because when we turned on the telescope, we had no training data. We had to look. But after we had spent a few years doing that, we had what was starting to look like a training set. I had that idea around the time Nabeel started working in our research group at Northwestern.

Are you using generative AI in your workflow?
Miller: We have a web page that collects and internalizes our collective knowledge. Someone essentially put a plugin for ChatGPT onto that particular web page. For any new explosion that we find, ChatGPT produces a three-sentence summary. Having generative AI that can develop these summaries and do it very quickly is really powerful and will be very helpful. Especially because, a couple of years from now, we're going to have a new telescope that's going to increase the rate at which we discover things by a couple of orders of magnitude.

You mentioned how using BTSbot helps save information. Have there been other benefits to using AI for this work?
Miller: The robotic telescope we use to get the spectrum is a small telescope. In real dollars and cents, it’s cheap to take a spectrum with that telescope. If we create the machine learning model, and it says, “These are the 10 most interesting things,” and it turns out two of those things are entirely uninteresting, then that's fine. We don't look at that as like a huge negative cost. In fact, you could even say that that's a cost of doing business. 

But the biggest telescopes in the world—like the Keck telescopes in Hawaii or the Very Large Telescope (VLT) that's in Chile—these projects cost hundreds of millions of dollars to build. Their equivalent nightly value is, roughly, $100,000. That's what the data is worth from one night on one of these telescopes. If you get to look at eight things and mess up one, that's kind of like setting [about] $15,000 on fire. But what's really nice about this work is that it's setting the groundwork for us to trust the system to make that very expensive decision at some point in the future.

How do you envision your work being used by future researchers?
Rehemtulla: This is something Adam and I think about quite frequently—making it easier to spin up new models and adapt existing ones. One of the things I've kept in mind during the development of the BTSBot is making the code that I write as flexible as possible.

I want it to be super easy to take it and put it on to the next task with minimal repeated effort. I want everything that is common between this and the next thing to be used. I don't want my work to go to waste. Similarly, because this code is public, that also means other people can use it and adapt it to whatever they may want to do. Making these tools public and well-documented makes it a little easier for people to get in. 

What advice would you give to people who are curious but hesitant about automating their work?
Miller: I run an educational program for graduate students called ‌the Data Science Fellowship Program. The unofficial mantra of that particular program has become: worry about the data. 

The pitfalls in astronomy are not that the AI doesn't work, or that we should [worry whether] the AI is trustworthy or anything like that. I think the major pitfall that can happen is that people don't actually understand their training sets before they start spinning stuff up. 

A lot of people talk about doing something that's unbiased. That literally cannot be true in astronomy because at some point, someone decided they're going to point their telescope over there. Whatever decision led to someone pointing their telescope there instead of there is a bias. Maybe that's not a meaningful bias depending upon whatever the science is that you're trying to do. But I sort of yell at people, like, don't worry too much about the machine learning. At some point, you'll put the data in some format. You'll pick some machine learning model, and then you'll get some results and maybe you pick another model and the results get a little bit better. 

You're way more likely to run into trouble if you haven't been worrying about the data, if you haven't been understanding what your real starting point is. Because I do think that is just so critical in terms of understanding what comes out on the other side.

Looking ahead, what’s the next task you’d like to automate?
Rehemtulla: Finding these explosions as fast as possible. It's not necessarily saving us time. But it's collecting these spectra at times when humans usually would not get them. It's opening new doors rather than saving time in existing ways. 

Miller: If these models could get to the point where we became more confident in asking more precise questions, instead of just asking, “Is this going to become a bright supernova?” We could ask, “Is this explosion the result of a supermassive black hole that’s destroyed and is now eating a star?” If we could answer that, I'm super excited.

This interview has been lightly edited and condensed for clarity.