TOPS logo

NASA TOPS—Success Stories of Open Science Series

Equitable science for all: Q&A with Dr. Flavio Azevedo on open science practices

July 25, 2022
By Steffie S. Kim [Email] [Twitter]
Digital Marketing Intern at NASA Transform to Open Science
*Session Mentor: Isabella B. Martinez

Dr. Flavio Azevedo

Dr. Flavio Azevedo is a political psychologist and an associate researcher at the University of Cambridge. Primarily, his research investigates the role of ideology and identity in supporting policies that perpetuate social and economic injustices. Having come from a low socioeconomic status in Brazil, he fell in love with the promise of academia as a great equalizer. This passion has led him to co-found and direct the Framework for Open and Reproducible Research Training (FORRT), an award-winning interdisciplinary community of almost 500 early-career scholars, aiming to integrate open scholarship principles into higher education and advance research transparency, reproducibility, rigor, and ethics through pedagogical reform and meta-science. In addition, he is participating in the NASA TOPS’ effort to build the OpenCore curriculum.

What is your definition of open science?
While definitions vary, there are usually six components to open science: open methodology, open data, open access, open peer review, open education, and open collaboration. But, for me, open science is about social justice—the inclusion of disenfranchised folks and building a coalition. It paves the way toward a more transparent, rigorous, robust, inclusive, diverse, accessible, and equitable science.

Essentially, open science means better science for all, including people of low and middle-income countries. Currently, there is a huge gap between how science benefits the Global North and Global South. For a rather recent example, we witnessed this during the pandemic, especially with the distribution of the COVID-19 vaccines, but examples abound across time and space. It is about breaking down barriers in accessing science, not only its outputs but its many benefits. It is about breaking that glass ceiling that pervades everything in science and research: That veneer of exclusiveness, of a selected club or that of the genius scholar, that only the best and brightest can ever dare to dream of becoming a researcher and succeeding in it. Taken together, open science is about social justice, and I practice it to leave behind a world that’s a tiny bit better than the one I found. So, I think open science is science done right.

At FORRT, we use the phrase “open scholarship” instead of open science. Folks talk about open scholarship and open science as synonyms. But I think open scholarship is more of a redefinition and reframing. Open scholarship is more inclusive in that it extends open science to all knowledge systems, including those not traditionally identified as science. Open scholarship also includes mentoring, teaching, and producing educational materials. And open scholarship particularly makes explicit the importance of inclusion, diversity, equity, and accessibility as necessary conditions for improving the way we practice science. This is very important to me because open scholarship becomes a tool to develop strategies for addressing structural disadvantages faced by minoritized groups. So, open scholarship is a way of looking at open science from a more humanistic view.

What steps are you taking to accelerate open science and open scholarship?
As the director of FORRT, I try to steer the organization’s ethos alongside other like-minded folks to achieve three things. The first is to respond to calls to consider open scholarship as inclusive scholarship. The second is to raise awareness of the pedagogical implications of open science/open scholarship and its associated challenges (i.e., curricular reform, epistemological uncertainty, methods of education, epistemological pluralism). The third is to reframe the “methodological reform” debate in academia as an opportunity to consider individual and systemic factors when evaluating research and the norms that sustain it.

Let me give you a few examples. We believe there is a need to reform how we teach and mentor our students. In social sciences, we rely too much on teaching the facts of science instead of focusing almost exclusively on the processes by which a given knowledge was acquired. In light of the reproducibility crisis (or credibility revolution, to put a positive spin on it), it is likely better for students if higher education focused more on scientific literacy (i.e., what does robust research look like?) and less on the facts of science. Another idea to consider is epistemological pluralism. There are a plethora of methods for acquiring knowledge; quantitative methods are not the only way. There are also qualitative ways. We need to recognize that there are multiple ways to go about knowledge acquisition and accumulation.

Aside from the initiatives at FORRT, as an independent researcher, I also conducted research using preregistrations and registered reports. I am part of several open science organizations, such as the Center for Open Science (COS) as an ambassador, Psychological Science Accelerator as a funding and finance committee member and researcher, and Berkeley Initiative for Transparency in the Social Sciences as a catalyst for open science. I also participated in big-team science like COS’ Systematizing Confidence in Open Research and Evidence (SCORE),  International Consortium for Social and Moral Psychology, the Crowdsourced Replication Initiative, and more recently at ManyLabs on Climate Change attitudes and Trust in Science. I think my approach to open scholarship is not only FORRT or reproducing things but also “interrogating the questions we pose,” as my dear colleague and feminist scholar, Madeleine Pownall, says.

What challenges have you faced while practicing open science or open scholarship? What strategies did you use to overcome these challenges?
I have faced a lack of funding, both for my research and for producing open educational resources as the director of FORRT. We have folks who provide unpaid contributions to the public good. We are constantly seeking institutional and financial support, but, unfortunately, we are often faced with a scarcity mindset across several academic and funding institutions. This is a big challenge in big-team science, especially when it comes to working with volunteer-based communities.

The other issue has to do with equitable authorship and credit. Credit is usually given in an unfair or non-credible way. In big-team science, for example, you see research projects where only Americans are the leaders, and they get most of the credit. And then, authorship becomes embedded into the current system of incentives and not a tool for mitigating inequalities in research, education, and funding schemes. Even in open science, there are not enough initiatives that try to include the Global South and low-income countries, or even minoritized groups in the U.S. You see the same patterns when it comes to credit-giving—it's not equitable. I find that troubling, and it is a challenge for me as an immigrant and a non-native English speaker. My work is devalued based on my identity. (A lot of the identities that we have are not internal. They come from external stereotypes.) Thankfully, there are at least two newly founded organizations aiming to bring light to these issues: Advancing Big-team Reproducible science with Increased Representation (ABRIR), led by Dr. Nadia Corral-Frías and colleagues, and NowhereLab, led by Dr. Priya Silverstein and colleagues.

At FORRT, we try to use a contributorship model based on the CRediT system as a means to document fairly every contribution to our research outputs. FORRT projects also try to include everybody by being proactive in recruiting contributors from the Global South and minoritized groups. Some of these contributors, depending on their conditions, might not be able to contribute a lot but we still appreciate the time that they can give to our projects. Using this inclusive model, FORRT produced over 15 large-scale Open Educational Resources (OER) and several peer-reviewed publications, including a paper in Nature Human Behavior with 112 authors, where we defined over 250 Open Science terms (see OA postprint here). While we have papers with few authors, the majority have close to 50 authors. I'm not saying that all papers should have that many authors. However, often in big-team science and regular science as well, there’s lots of invisible work that is not credited, and we're tracking and giving credit to every meaningful contributor because big and meaningful educational and scientific projects need contributions from a lot of folks and it is important (and just!) to recognize it.

Can you elaborate on the credit-giving system you are using at FORRT?
We usually use Tenzing, which is essentially a Google Sheets document where everybody can enter their contact information and their contributor roles. There are 14 types of contributions as per the CRediT system—sometimes we include a few more—which tracks how much people have participated in a particular project. For example, we track idea formulation, conceptualization, or project administration in each column. It's a very nifty tool that enables you to give credit for what people did.

Another thing I want to share is how we went about our work that aimed to bridge neurodiversity and open scholarship (see page 86 in this hyperlinked citation and the figure below). What we essentially tried to do here was to find a different way to think of authorship. The authorship for this paper was not decided based on the authors' contributions but rather on their privileges. We put the folks who are least privileged first, as this brings the most prestige in academia. We adopted a critical lens about how we give authorship because, often, there is a privilege in your ability to contribute—you need to have had successive opportunities, time, and funding, for example. So, this is a way to provide an alternative way to give credit, contingent upon one’s privileges. Maybe this is not the right way, but when all you have is the current system—which is gamified, unfair, and not transparent—we thought that providing an alternative is the first step forward.

Academic Wheel of Privilege, a color wheel made up of many factors. Least privileged factors (homeless, dark-skinned, trans, etc) are on the outside and most privileged (rich, white, hetero, etc) are on the inside.

Have you experienced failure while practicing open science? How would you address them differently if you were to do it again?
I came to open science from a methodological, statistical, and even a little dogmatic point of view. Having a community, exchanging knowledge, and taking on other people’s perspectives allowed me to see further. If I could start FORRT again, I would make it an early focus to find funding for marginalized people and those who are trying to give their best to the community. I was not aware of the institutions that I had to contact or the grant applications I had to apply for so that we would have ways to compensate folks. I wish I had been better at communicating and understanding our community members and the system of incentives. Shout out to Sam Parsons, Amy Orben and CERES, Thomas Rhys Evans, Madeleine Pownall, Jackie Thompson, and folks at Invest in Open Infrastructure (IOI), who are helping FORRT to do better in this regard!

How has open science improved your research? Are there other benefits you have experienced from practicing open science?
I have benefited immensely from getting to know folks around the globe. Being open to learning from different perspectives has been the most valuable benefit. Open Science also helped me connect my “regular” research, which is essentially on justice, with better research practices. I’m a political psychologist by training, and the main question driving my research is: Why do some believe that a nation, people, race, gender, or species is justified in dominating, controlling, and exploiting another? Practicing open science has also opened my eyes to the inequities of academia and how we conduct research in a very non-inclusive and non-participatory way. And I hope to contribute to changing this.

What are your recommendations for practicing open science? What are the most important things that should be addressed in the field to accelerate open science further?
One aspect of open science is often absent from the discussion: the pedagogical consequences of how we teach, mentor, and supervise students through open scholarship. The future of education requires open scholarship principles to be integrated into research training. I think that pedagogical communities in that integration play a significant role, especially in this super-connected world. Pedagogical communities can help co-create materials available for everyone while fostering an inclusive community across all career stages, diverse disciplines, and different regions. Pedagogical communities also offer a low entry point for research and better practices in a much-needed environment. People can identify common hurdles by exchanging opinions. Pedagogical communities are definitely something that we should pay more attention to and recognize. FORRT folks have published a manuscript entitled, Towards a culture of open scholarship: The role of pedagogical communities, discussing this further. The benefit of pedagogical communities and the role they play in fostering an inclusive culture of open scholarship and calling for greater collaboration with pedagogical communities to pave the way for a much-needed integration of top-down and grassroots open scholarship initiatives.

What would you say to early career researchers who want to practice open science?
Admittedly, what I'm about to say is a very narrow and personal view because I think there are so many more qualified folks that could provide a plethora of helpful answers. But, I would say that joining a community that speaks to your heart and your science would be the best way to go. That community can be any open science community, whether it is about sharing data, creating education materials, or conducting research projects. Through community, we learn how to best conduct open scholarship on a day-to-day basis. Especially for early career scholars from the Global South, having a community to talk about issues, ask for help, and see other people asking for help can have a positive effect.

What are some of your favorite open science tools or resources that you’d like to share with us?
From a social science perspective, I like pedagogical communities because they are essentially trying to provide a common good for others. The Turing Way is one of my favorites—it is a very well-run organization. Also, Open Life Science, The Carpentries, and Berkeley Initiative for Transparency in Social Sciences (BITSS) are great. The RIOT Science Club offers seminars where people can learn about open science, open scholarship, and new methodologies. A few projects try to implement replication, such as The Institute for Replication. These are major institutions, but there are so many great researchers like Charlotte Pennington, Julia Strand, Gilad Feldman, Katie Button, Lisa DeBruine, and Jordan Wagge that I try to mirror.

Regarding open tools, I use OSF, Zenodo, and SHARE. CRediT and Tenzing are a really good way to give fair credit to people based on what they did on a given paper. You can use it on a Shiny app that exports all the information neatly, which helps a lot when writing papers with dozens of co-authors. As I mentioned about communities, the R Studio Community is great. They are amazing folks with amazing open products. I also want to give a shout-out to R-Ladies, which is an amazing group.

Lastly, is there anything you would like to share regarding open science?
A few weeks ago, I shared with my friends and colleagues on Twitter that I am going through a medical problem. I got overwhelming supportive reactions from folks. But, the thing that struck my heart the most was the messages I got from people who shared their own stories about their disability and how in academia, it is a reason for shame—something that should be hidden. And even people that are extremely well-known in my field sent me direct messages about their struggles. This made me sad. We internalize the invisibleness and powerlessness across every tier of academia. On top of dealing with a debilitating condition, disability becomes a source of shame and weakness. We need to do more in this regard, be more open, and normalize these struggles we go through in life. We need to talk more about disability in academia.