Churchland lab

James Roach and Simon Musall, two postdocs in my lab, took the lead on this write-up.

This new article by Adam Calhoun, Jonathan Pillow & Mala Murthy leverages detailed behavioral data during courtship of fruit flies to demonstrate that sensorimotor transformation is highly state dependent. The authors combine a hidden-markov-model (HMM) and generalized linear models (GLMs) to automatically identify different internal states from behavioral data. Each state has its own rules (transfer functions) that govern how sensory inputs drive different types of fly songs. Lastly, they use optogenetic stimulation to test if song-promoting neurons might instead be causing specific internal state transitions.

Big Question: How can we identify different internal states and understand how they shape the transformation from sensory input to motor output? This is a question that goes far beyond flies and has. broad relevance. 

Approach: The authors analyze 2765 minutes of fly movies featuring 276 fly couples. Using automated methods, they identify a bunch of behaviors called ‘feedback cues’ that the male flies could get from themselves and a female partner. To identify internal states, they developed a GLM-HMM that predicts different mating songs based on feedback cues and song history. The model was able to predict held out data really well, far better than a traditional GLM approach. In contrast to a traditional HMM, the GLM-HMM also uses feedback cues to determine when to switch between internal states and was also much better in predicting when flies transition between different song types.

The authors then dug deep to better understand what defines the states and how they differ in terms of the relationship between feedback cues and song production. They find that states aren’t simply defined by the current song or the incoming feedback cues. Instead, what defines a state is the exact relationship between the feedback cues and song production: each state could produce diverse song outputs, but the ways that feedback cues predicted which song was produced were largely different across states.

Finally, they optogenetically manipulated 3 cell types in the brain and observed that stimulation of one type, PiP10, drove the animal into a “close” state. There is a subtlety here: the animal didn’t just sing more, instead, switching internal states made some song types in response to particular feedback cues more likely, ruling out a much simpler model (a summary is below).

Take homes:

1. Sensorimotor transformation is highly state dependent and different feedback cues can lead to state changes.
2. One can identify states form behavioral data alone in an unsupervised way. These differ from experimenter-imposed states like hunger and satiety because the animal engages in them voluntarily, and switches states on a fairly rapid timescale (e.g., seconds).

3. States can have highly overlapping behavioral repertoires, both in terms of the sensory cues that are present and the song outputs that are observed.

4. Behavioral states are not fixed, as we often assume, but vary continuously. What is really novel here is that they used an HMM to identify latent states, as opposed to experimenter defined ones like satiety and hunger. Assuming an animal is in a fixed state throughout an experiment can lead us astray and we can miss important information about how animals interact with their environments.

Skeptics’ corner:

We were surprised that the addition of so many new parameters doesn’t improve performance of the GLM-HMM relative to the HMM more. A closer comparison between the HMM and GLM-HMM, (e.g. in Fig. 3a) would have helped us understand how the addition of state-dependent emission GLMs improves sequence prediction compared to a fixed emission HMM. Also, autocorrelations seem to be a strong factor in the success of the HMM and mixed models. It would be interesting to see how the standard GLM would perform when an autoregressive term is added to it. 

Activation of PiP10 promotes a ‘close’ state transition and yet the animal does LESS of the sine song. This is intriguing because the sine song is the most probable output in the close state state, so this divergence seems counter-intuitive (Fig. 2). In a way, this is exciting! It reiterates that the fly is in a state NOT because of what it is doing now, but because of how the feedback cues shape the behavior. But we still found the magnitude of that difference confusing. In a related point, how do males behave beyond song production in each state? Does PiP10 stimulation lead to the male moving as if it is in the “close” state even if they are far away from the female?

Manipulating neural activity to induce state transitions will likely be a widely used and informative probe into animal brain states. Interestingly, this will lead to brain states that are inappropriate for a given context. We think of this as being a bit like a multisensory “conflict condition”: the brain is telling the animal it is in one state, but everything around the animal (e.g., its distance from the female) might be more consistent with a different state. How should we be thinking about the fact that the optogenetics push the animal into a conflict condition? Is this an off-manifold perturbation? 

Outlook:

The term ‘feedback cues’ combines self-initiated components like male velocity with externally-imposed components like female velocity. It would be interesting to separate those out further to better understand these different components influence state-transitions and song production. Functionally grouping ‘feedback cues’ might also provide additional insight into which features they influence the most.

More emphasis on state-transition GLMs would be very interesting to better understand how transition are guided by sensory feedback cues. The kernels shown in Fig. S4 indicate different patterns of high ethological significance. Highlighting these more would further demonstrate the usefulness of the GLM-HMM approach in general.

We wished there were a low-dimensional summary that allowed us to more easily visualize what the collection of behaviors were in each state. This maybe underscores a general problem in the field which is when you probe behavior with unsupervised learning tools, you end up with results that are deeply informative, and very powerful, but hard to summarize. We struggled with this as well, when connecting video to neural activity using unsupervised methods. I’m hoping folks will have emerging ideas about how to do this.