Episodes
-
In this episode, Brian Houck, Applied Scientist, Developer Productivity at Microsoft, covers SPACE, DORA, and some specific metrics the developer productivity research team is finding useful. The conversation starts by comparing DORA and SPACE. Brian explains why activity metrics were included in the SPACE framework, then dives into one metric in particular: pull request throughput. Brian also describes another metric Microsoft is finding useful, and gives a preview into where his research is heading.
Mentions and links
Connect with Brian on LinkedInThe SPACE of Developer Productivity: There's More to It Than You ThinkMeasuring developer productivity with the DX Core 4DevEx in action paperDORA, SPACE, and DevEx: Which framework should you use?Discussion points
(0:48) SPACE framework's growth and adoption(3:47) Comparing DORA and SPACE(6:30) SPACE misconceptions and common implementation challenges(9:34) Whether PR throughput is useful (15:13) Real-world example of using PR throughput (21:33) Talking about metrics like PR throughput internally (24:39) Where Brian’s research is heading -
Click here to view the episode transcript.
In this episode, Snowflake’s Gilad Turbahn, Head of Developer Productivity, and Amy Yuan, Director of Engineering, dive into how they elevated developer productivity to a top company priority. They discuss the pivotal role of Snowflake’s CTO, who personally invested over half his time to guide the initiative, and how leadership's hands-on involvement secured buy-in across teams. The conversation also explores the importance of collaboration between engineering and product management, and how measuring user sentiment helped them deliver meaningful, long-lasting improvements.Mentions and links
Connect with Gilad and Amy on LinkedInMeasuring developer productivity with the DX Core 4Discussion Points
(0:48) The need for a shift at Snowflake(3:59) Leadership involvement and prioritization of developer productivity(8:56) The partnership between engineering and product managers(20:01) From feature factory to customer outcome-focused development(27:36) Shifting measurement focus to user sentiment and customer outcomes(39:13) Gaining buy-in for sentiment metrics and tying them to business impact(51:11) How Snowflake’s CTO and volunteers accelerated developer productivity improvements. -
Missing episodes?
-
Click here to view the episode transcript.
In this episode, Emanuel Mueller Ramos, Head of Developer Experience at Skyscanner, discusses the evolution of his team as they transitioned from focusing on frameworks and middleware to becoming a customer-centric, impact-driven organization. Emanuel details the strategies he used to gain stakeholder buy-in, why it's crucial to rethink traditional productivity metrics, and how they made a cultural shift to prioritize developer happiness and effectiveness. This conversation highlights the steps necessary to build a developer experience function that delivers meaningful impact.
Follow Emanuel on LinkedInMeasuring developer productivity with the DX Core 4
Mentions and links:Discussion points:
(1:14) The beginning of Skyscanner's developer productivity division(3:53) Gaining stakeholder buy-in and refocusing the teams(5:57) Redefining success metrics for developer productivity(8:57) Pitching the developer experience focus to leadership(17:26) Moving from frameworks to feedback loops(20:45) Fostering a customer-centric culture(23:20) Defining the collaboration between platform and developer experience teams(26:41) Choosing the right metrics for developer experience success (31:31) Risks and challenges ahead -
Click here to view the episode transcript.
In this week's episode, Abi is joined by industry leaders Idan Gazit from GitHub, Anna Sulkina from Airbnb, and Alix Melchy from Jumio. Together, they discuss the impact of GenAI tools on developer productivity, exploring challenges in measurement and enhancement. They delve into AI's evolving role in engineering, from overcoming friction points to exploring real-world applications and the future of technology. Gain insights into how AI-driven chat assistants are reshaping workflows and the vision for coding.
How to measure GenAI adoption and impact
Links:Timestamps:
(2:58) Challenges of Measuring AI Productivity(6:02) Use cases for GenAI within the Airbnb developer organization(10:26) GitHub’s process for developing and testing new GenAI tools for developers(12:42) Driving GenAI adoption strategies at Airbnb(14:20) Research impact and productivity gains with GenAI tools at Airbnb(17:03) Copilot use cases surveyed among Jumio's developers(18:46) Challenges measuring impact of AI products at GitHub(21:33) Biggest gains of GenAI usage at Airbnb(24:19) Future opportunities in GenAI(30:31) Challenges in GenAI for developers -
Click here to view the episode transcript.
In this week's episode, Abi welcomes Jared Wolinsky, Vice President of Platform Engineering at SiriusXM, to delve into the inner workings of platform engineering at SiriusXM. Jared sheds light on their innovative approach to prioritizing projects, emphasizing alignment with overarching business goals. They explore how these strategies boost developer speed and drive technological advancement within the organization.
When is the right time to establish a DevProd team report
Links:Timestamps:
(1:39) SiriusXM's major rebuild(4:46) Challenges of building a platform during a major product revamp(7:22) Navigating trade-offs(10:06) Defining the ideal developer journey at SiriusXM(17:28) Navigating the path to a user-centric developer experience(23:05) Collaborating with leadership to iterate and gain approval(25:05) Balancing enablement and platform(28:28) Implementing a data-driven prioritization framework(34:29) Aligning projects with business goals -
Click here to view the episode transcript'.
In this episode, Michelle Swartz, Vice president of Developer Enablement American Express, shares insights on improving developer experience. She discusses the creation of an onboarding bootcamp and the development of the AmEx Way Library for better knowledge management. Michelle explains how AmEx balances standardization and flexibility with the concept of Paved Roads. She also highlights the importance of measuring success, fostering community, and elevating the company's tech credibility.Mentions and links
GenAI GuideTimestamps
(5:45) Challenges of advocating for DevEx in non-tech companies(7:43) Importance of senior leadership buy-in for DevEx(9:58) Genesis of the DevEx organization and Jedi Council(12:12) Transition to a dedicated DevEx function(13:17) Formalizing investment in DevEx(18:02) Initial efforts and learning in improving DevEx(19:25) Using sentiment surveys to prioritize DevEx areas(27:26) Addressing knowledge management challenges(29:49) Balancing standardization and freedom in DevEx(36:21) Implementing Paved Roads: evolution vs. revolution -
Click here to view the episode transcript'.
This week’s episode is a recording from a recent event hosted by Abi Noda (CEO of DX) and Laura Tacho (CTO at DX). The episode begins with an overview of the DORA, SPACE, and DevEx frameworks, including where they overlap and common misconceptions about each. Laura and Abi discuss the advantages and drawbacks of each framework, then discuss how to choose which framework to use.Mentions and Links:
Dora.devDiscussion points:
2:50- DORA, SPACE, DevEx overview10:35- Choosing which framework to use13:15- Using DORA22:42-Using SPACE -
Click here to view the episode transcript.
In this week's episode, we welcome Derek DeBellis, lead researcher on Google's DORA team, for a deep dive into the science and methodology behind DORA's research. We explore Derek's background, his role at Google, and how DORA intersects with other research disciplines. Derek takes us through DORA's research process step by step, from defining outcomes and factors to survey design, analysis, and structural equation modeling.Mentions and Links:
Derek DeBellis on LinkedInDX’s guide to measuring GenAI adoption and impact2023 Accelerate State of DevOps ReportDiscussion points:
(3:00) Derek’s transition from Microsoft to the DORA team at Google(4:28) Derek talks about his connection to surveys(6:16) Derek’s journey to becoming a quantitative user experience researcher(7:48) Derek simplifies DORA(8:19) DORA - Philosophy vs practice(11:09) Understanding desired outcomes(12:45) Self reported outcomes vs objective outcomes(16:16) Derek and Abi discuss the nuances of literature review(19:57) Derek details survey development(27:55) Pretesting issues(29:30) Designing surveys for other companies(35:02) Derek simplifies model analysis and validation techniques(38:48) Benchmarks: Balancing data limitations with method sensitivity -
Click here to view the episode transcript.
This week we’re joined by Sean Mcllroy from Slack’s Release Engineering team to learn about how they’ve fully automated their deployment process. This conversation covers Slack’s original release process, key changes Sean’s team has made, and the latest challenges they’re working on today.Mentions and links:
Read Sean’s blog post, The Scary Thing About DeploysFollow Sean on LinkedInTime Stamps:
(1:34): The Release Engineering team(2:13): How the monolith has served Slack (3:24): How the deployment process used to work (6:23): The complexity of the deploy itself(7:39): Early ideas for improving the deployment process(9:07): Why anomaly detection is challenging(10:32): What a Z-score is(13:23): Managing noise with Z-scores(16:49): Presenting this information to people that need it(19:54): Taking humans out of the process(23:13): Handling rollbacks(25:27): Not overloading developers with information(28:26): Handling large deployments -
Click here to view the episode transcript.
This week’s episode is the recording of a live conversation between Abi and Chris Westerhold (Thoughtworks Head of Developer Experience). This conversation is useful for anyone early in their journey with developer portals or platforms: Abi and Chris discuss common approaches to solving these problems, pitfalls to avoid, building vs. buying, and more.Mentions and Links
Follow Chris on LinkedInWatch the recording of this conversation Watch part 2 of this conversation on the market landscapeLearn about PlatformX, DX’s product mentioned in the conversationTime Stamps:
(3:09) Why there’s an increased interest in developer portals(5:33) Chris’ background with dev portals(6:37) Homegrown solutions for developer portals(9:22) How developer portal initiatives begin(11:24) Internal developer portal vs service catalogs and IDPs(16:18) Mistakes companies make with developer portals(21:05) Approaches to solving this problem(24:28) How can developer portals drive value(32:07) Common traps to avoid -
Click here to listen to the episode transcript.
On this week's episode, Abi interviews Kent Wills, Director of Engineering Effectiveness at Yelp. He shares insights into the evolution of their developer productivity efforts over the past decade. From tackling challenges with their monolithic architecture to scaling productivity initiatives for over 1,300 developers. Kent also touches on his experience in building a business case for developer productivity.
(1:42) Forming the developer productivity team(3:25) Naming the team engineering effectiveness(4:30) Getting leadership buy-in for focusing on this work(7:54) Managing code ownership in Yelp’s monolith(12:23) Supporting the design system(16:00) The business case for forming a dedicated team (19:45) How to standardize (23:50) How their approach to standardization might be different in another company(27:08) Demonstrating the value of their work (32:21) Building an insights platform(38:47) How Yelp is using LLM’s
Discussion points:Mentions and Links
Connect with Kent Wills on LinkedInWatch Kent’s 2023 talk at ElevateListen to the interview with Peter Seibel (“Let 1,000 flowers bloom”)Download the recently published benchmarks on developer productivity team headcount -
This week we’re joined by Gail Carmichael, Principal Instructional Engineer at Splunk. At Splunk, Gail’s team is responsible for improving developer onboarding, which they do through a multi-day learning program. Here, Gail shares how this program works and how they measure developer onboarding. The conversation also covers what instructional engineers are generally, and how Gail demonstrates the impact of her team’s work.
(1:16) The Engineering Enablement & Engagement Team at Splunk(8:01) What an Instructional Engineer is(14:36) The developer onboarding program at Splunk(16:05) Components of a good onboarding program(21:11) Why having an onboarding program matters(28:17) Measuring onboarding at Shopify (Gail’s previous company)(31:39) Measuring developer onboarding at Splunk
Discussion points:Mentions and Links
Connect with Gail on LinkedInDownload the report on Developer productivity metrics at top tech companies -
In this episode we’re joined by Adam Rogal, who leads Developer Productivity and Platform at DoorDash. Adam describes DoorDash’s journey with their internal developer portal, and gives advice for other teams looking to follow a similar path. Adam also describes how his team delivered value quickly and drove adoption for their developer platform.
Discussion points:
(1:47) Why DoorDash explored implementing a developer portal(6:59) The initial vision for the developer portal 12:19 Funding ongoing development 16:01 Deciding what to include in the portal 19:15 Coming up with a name for the portal 20:01 Advice for interested beginners23:55 Putting together a business case32:32 Getting adoption for the portal 37:27 Driving initial awareness 41:29 Getting feedback from developers48:33 What Adam would have done differentlyMentions and links:
Adam Rogal on LinkedInGet started (API)New testing and monitoring tools -
In this episode, Abi has a fascinating conversation with Rebecca Parsons, ThoughtWorks's CTO, Camilla Crispim, and Erik Dörnenburg on the ThoughtWorks Tech Radar. The trio begins with an overview of Tech Radar and its history before delving into the intricate process of creating each report involving multiple teams and stakeholders. The conversation concludes with a focus on the evolution of Tech Radar's design and process and potential future changes. This episode offers Tech Radar fans an exclusive behind-the-scenes look at its history and production.
Discussion points:
1:20-An introduction to the Tech Radar6:06-Common terms used in this episode6:27-The origin of the Tech Radar8:50-Problems that the Tech Radar was aiming to solve12:23-The impact on internal decision making-a tool for driving change14:30-The teams philosophy behind Tech Radar18:33-What sets the Tech Radar apart21:11-Why maintaining independence is crucial for their audience25:08-How Tech Radar publishes their reports29:36-A look into Thoughtworks live meeting sessions34:51-Tech Radars Git repository42:20-Recent changes and upcoming shiftsMentions and links:
ThoughtWorks TechRadarRebecca Parsons on LinkedInCamilla Crispim on LinkedInErik Dörnenburg on LinkedInThoughtworks Git repository -
This week's guest is Eirini Kalliamvakou, a staff researcher at GitHub focused on AI and developer experience. Eirini sits at the forefront of research into GitHub Copilot. Abi and Eirini discuss recent research on how AI coding assistance impacts developer productivity. They talk about how leaders should build business cases for AI tools. They also preview what's to come with AI tools and implications for how developer productivity is measured.
(1:49) Overview of GitHub’s research on AI(2:59) The research study on Copilot(4:48) Defining and measuring productivity for this study(7:44) Exact measures and factors studied(8:16) Key findings from the study(9:45) How the study was conducted (11:17) Most surprising findings for the researchers(14:01) The motivation for conducting a follow-up study(15:34) How the follow-up study was conducted(18:42) Findings from the follow-up study(21:13) Is AI just hype? (26:34) How to begin advocating for AI tools(34:44) How to translate data into dollars(37:06) How to roll out AI tools to an organization(38:47) The impact of AI on developer experience(43:24) Implications of AI on how we measure productivity
Discussion points:Mentions and links:
Eirini Kalliamvakou on LinkedInResearch on the impact of Copilot Crossing the Chasm by Geoffrey Moore -
Christopher Sanson is a product manager at Airbnb who is dedicated to enhancing developer productivity and tooling. Today, we learn more about Airbnb's developer productivity team and how various teams use metrics, both within and outside the organization. From there, we dive even deeper into their measurement journey, highlighting their implementation of DORA metrics and the challenges they overcame throughout the process.
Discussion points:
(2:43) Who is the developer productivity customer(4:49) The evolution of developer productivity at Airbnb(9:26) Approach before DORA metrics(14:29) Getting buy-in for DORA metrics(17:49) Planning how to deliver new metrics to the organization(21:12) How Airbnb calculates deployment frequency(23:29) Implementing a proof of concept(27:20) Statistical measurement strategies and tactics(31:11) Operationalizing developer productivity metrics(34:26) How Airbnb reviews data(35:41) How Airbnb uses DORA metricsMentions and links:
Christopher Sanson on LinkedInChristopher’s talk at DPE SummitHow Top Companies Measure Developer Productivity -
In this episode, Abi speaks with Ana Petkovska, who is currently leading the developer experience team at Nexthink. Ana takes us through her journey of leading a DevOps team that underwent multiple transformations. She explains how her team went from being a DevOps team to EngProd and eventually DevEx. Ana elaborates on her team's challenges and the reasons behind the shift in focus. She also shares how she discovered EngProd and used data from companies like Google to convince her company to invest in EngProd. Finally, Ana explains how DevEx came into the picture and changed how her team approaches and measures their work.
Discussion points:
(00:28) Creating and leading a DevOps team(05:04) Shifting from DevOps to EngProd(07:28) Inspiration from Google(10:05) Building the case for EngProd(13:42) Ratio of engineers to DevEx engineers(15:10) Team mission and charter(16:53) Learning about DevEx(20:05) The difference between EngProd and DevEx(22:32) Nexthink's focus todayMentions and links:
Ana Petkovska on LinkedInEngineering Productivity @Google (Michael Bachman) -
In this episode, Abi chats with Grant Jenks, Senior Staff SWE, Engineering Insights @ LinkedIn. They dive into LinkedIn's developer insights platform, iHub, and its backstory. The conversation covers qualitative versus quantitative metrics, sharing concerns about these terms and exploring their correlation. The episode wraps up with technical topics like winsorized means, thoughts on composite scores, and ways AI can benefit developer productivity teams.
(1:10) Insights in the productivity space
(7:13) LinkedIn's metrics platform, iHub
(12:52) Making metrics actionable
(15:35) Choosing the right and wrong metrics
(19:39) The difficulty of answering simple questions
(26:23) Top-down vs. bottom-up approach to metrics
(32:12) Winsorized mean and selecting measurements
(39:25) Using composite metrics
(46:57) Using AI in developer productivity -
This week's episode is with Jim Beyers, VP of Engineering Enablement at CVS Health. Jim joined CVS a year ago to lead an effort to build an internal developer platform. Abi and Jim discuss how Jim joined CVS to build an internal developer platform, what brought him to the job, and how the developer experience fits into the broader transformation goals of CVS. Additionally, this episode covers building the team, defining a strategy, and how he's thinking about winning the hearts and minds across his organization.
Discussion points:
(1:15) How Jim was brought into CVS(2:39) How DevEx aligns with CVS’s transformation initiatives(6:06) Jim’s vision for developer experience(8:26) Building a DevEx team and working with product managers(15:06) Defining and communicating a DevEx strategy(19:37) Assessing Backstage and developing a platform(24:40) Working with developers and leaders(27:55) Working alongside colleagues tackling similar problems(29:26) Reporting on progressMentions and links:
Jim Beyers on LinkedInJim’s talk on the Target Application Platform -
This week we spoke with Nils Loodin, Platform Product Manager at Spotify. Nils describes how his role in platform product management works, including unique challenges, approaches, and career considerations. Nils also discusses some of the recent changes within Spotify's platform organization, including shifting teams from tech-centric to journey-centric.
Discussion points:
(1:30) How Nils came into his role(3:59) How “developer experience” came into the picture at Spotify(5:30) How the Platform team is structured(8:52) Unique challenges of the Platform PM role(12:51) Defining the Platform PM’s focus(16:39) Staying close to their customers(21:09) Optimal background for someone in this role(24:43) Attracting PMs into Platform roles(29:40) How it is that Spotify’s leadership invests in developer experience(31:19) How a recent reorg shifted Platform’s focus (41:29) Improving onboarding for mobile engineers(47:33) Measuring onboardingMentions and links:
Connect with Nils on LinkedInThe product management discipline in platform teams | Russ Nealis (Plaid)Spotify’s Engineering blog - Show more
