The AI Researcher Giving Her Field Its Bitter Medicine – Quanta Magazine

Anima Anandkumar, Bren Professor of computing at the California Institute of Technology and senior director of machine learning research at Nvidia, has a bone to pick with the matrix. Her misgivings are not about the sci-fi movies, but about mathematical matrices — grids of numbers or variables used throughout computer science. While researchers typically use matrices to study the relationships and patterns hiding within large sets of data, these tools are best suited for two-way relationships. Complicated processes like social dynamics, on the other hand, involve higher-order interactions.

Luckily, Anandkumar has long savored such challenges. When she recalls Ugadi, a new year’s festival she celebrated as a child in Mysore (now Mysuru), India, two flavors stand out: jaggery, an unrefined sugar representing life’s sweetness, and neem, bitter blossoms representing life’s setbacks and difficulties. “It’s one of the most bitter things you can think about,” she said.

She’d typically load up on the neem, she said. “I want challenges.”

This appetite for effort propelled her to study electrical engineering at the Indian Institute of Technology in Madras. She earned her doctorate at Cornell University and was a postdoc at the Massachusetts Institute of Technology. She then started her own group as an assistant professor at the University of California, Irvine, focusing on machine learning, a subset of artificial intelligence in which a computer can gain knowledge without explicit programming. At Irvine, Anandkumar dived into the world of “topic modeling,” a type of machine learning where a computer tries to glean important topics from data; one example would be an algorithm on Twitter that identifies hidden trends. But the connection between words is one of those higher-order interactions too subtle for matrix relationships: Words can have multiple meanings, multiple words can refer to the same topic, and language evolves so quickly that nothing stays settled for long.

This led Anandkumar to challenge AI’s reliance on matrix methods. She deduced that to keep an algorithm observant enough to learn amid such chaos, researchers must design it to grasp the algebra of higher dimensions. So she turned to what had long been an underutilized tool in algebra called the tensor. Tensors are like matrices, but they can extend to any dimension, going beyond a matrix’s two dimensions of rows and columns. As a result, tensors are more general tools, making them less susceptible to “overfitting” — when models match training data closely but can’t accommodate new data. For example, if you enjoy many music genres but only stream jazz songs, your streaming platform’s AI could learn to predict which jazz songs you’d enjoy, but its R&B predictions would be baseless. Anandkumar believes tensors make machine learning more adaptable.

It’s not the only challenge she’s embraced. Anandkumar is a mentor and an advocate for changes to the systems that push marginalized groups out of the field. In 2018, she organized a petition to change the name of her field’s annual Neural Information Processing Systems conference from a direct acronym to “NeurIPS.” The conference board rejected the petition that October. But Anandkumar and her peers refused to let up, and weeks later the board reversed course.

Quanta spoke with Anandkumar at her office in Pasadena about her upbringing, tensors and the ethical challenges facing AI. The interview has been condensed and edited for clarity.

How did your parents influence your perception of machines?

In the early 1990s they were among the first to bring programmable manufacturing machines into Mysore. At that time it was seen as something odd: “We can hire human operators to do this, so what is the need for automation?” My parents saw that there can be huge efficiencies, and they can do it a lot faster compared to human-operated machines.

Was that your introduction to automation?

Yeah. And programming. I would see the green screen where my dad would write the program, and that would move the turret and the tools. It was just really fascinating to see — understanding geometry, understanding how the tool should move. You see the engineering side of how such a massive machine can do this.

What was your mother’s experience in engineering? 

 My mom was a pioneer in a sense. She was one of the first in her community and family background to take up engineering. Many other relatives advised my grandfather not to send her, saying she may not get married easily. My grandfather hesitated. That’s when my mom went on a hunger strike for three days.

As a result, I never saw it as something weird for women to be interested in engineering. My mother inculcated in us that appreciation of math and sciences early on. Having that be just a natural part of who I am from early childhood went a long way. If my mom ever saw sexism, she would point it out and say, “No, don’t accept this.” That really helped.

Spread the love

Leave a Reply

Your email address will not be published.

The AI Researcher Giving Her Field Its Bitter Medicine – Quanta Magazine

Anima Anandkumar, Bren Professor of computing at the California Institute of Technology and senior director of machine learning research at Nvidia, has a bone to pick with the matrix. Her misgivings are not about the sci-fi movies, but about mathematical matrices — grids of numbers or variables used throughout computer science. While researchers typically use matrices to study the relationships and patterns hiding within large sets of data, these tools are best suited for two-way relationships. Complicated processes like social dynamics, on the other hand, involve higher-order interactions.

Luckily, Anandkumar has long savored such challenges. When she recalls Ugadi, a new year’s festival she celebrated as a child in Mysore (now Mysuru), India, two flavors stand out: jaggery, an unrefined sugar representing life’s sweetness, and neem, bitter blossoms representing life’s setbacks and difficulties. “It’s one of the most bitter things you can think about,” she said.

She’d typically load up on the neem, she said. “I want challenges.”

This appetite for effort propelled her to study electrical engineering at the Indian Institute of Technology in Madras. She earned her doctorate at Cornell University and was a postdoc at the Massachusetts Institute of Technology. She then started her own group as an assistant professor at the University of California, Irvine, focusing on machine learning, a subset of artificial intelligence in which a computer can gain knowledge without explicit programming. At Irvine, Anandkumar dived into the world of “topic modeling,” a type of machine learning where a computer tries to glean important topics from data; one example would be an algorithm on Twitter that identifies hidden trends. But the connection between words is one of those higher-order interactions too subtle for matrix relationships: Words can have multiple meanings, multiple words can refer to the same topic, and language evolves so quickly that nothing stays settled for long.

This led Anandkumar to challenge AI’s reliance on matrix methods. She deduced that to keep an algorithm observant enough to learn amid such chaos, researchers must design it to grasp the algebra of higher dimensions. So she turned to what had long been an underutilized tool in algebra called the tensor. Tensors are like matrices, but they can extend to any dimension, going beyond a matrix’s two dimensions of rows and columns. As a result, tensors are more general tools, making them less susceptible to “overfitting” — when models match training data closely but can’t accommodate new data. For example, if you enjoy many music genres but only stream jazz songs, your streaming platform’s AI could learn to predict which jazz songs you’d enjoy, but its R&B predictions would be baseless. Anandkumar believes tensors make machine learning more adaptable.

It’s not the only challenge she’s embraced. Anandkumar is a mentor and an advocate for changes to the systems that push marginalized groups out of the field. In 2018, she organized a petition to change the name of her field’s annual Neural Information Processing Systems conference from a direct acronym to “NeurIPS.” The conference board rejected the petition that October. But Anandkumar and her peers refused to let up, and weeks later the board reversed course.

Quanta spoke with Anandkumar at her office in Pasadena about her upbringing, tensors and the ethical challenges facing AI. The interview has been condensed and edited for clarity.

How did your parents influence your perception of machines?

In the early 1990s they were among the first to bring programmable manufacturing machines into Mysore. At that time it was seen as something odd: “We can hire human operators to do this, so what is the need for automation?” My parents saw that there can be huge efficiencies, and they can do it a lot faster compared to human-operated machines.

Was that your introduction to automation?

Yeah. And programming. I would see the green screen where my dad would write the program, and that would move the turret and the tools. It was just really fascinating to see — understanding geometry, understanding how the tool should move. You see the engineering side of how such a massive machine can do this.

What was your mother’s experience in engineering? 

 My mom was a pioneer in a sense. She was one of the first in her community and family background to take up engineering. Many other relatives advised my grandfather not to send her, saying she may not get married easily. My grandfather hesitated. That’s when my mom went on a hunger strike for three days.

As a result, I never saw it as something weird for women to be interested in engineering. My mother inculcated in us that appreciation of math and sciences early on. Having that be just a natural part of who I am from early childhood went a long way. If my mom ever saw sexism, she would point it out and say, “No, don’t accept this.” That really helped.

Spread the love

Leave a Reply

Your email address will not be published.