[Oss-health-metrics] Metric relevant topics during CHAOSS breakout session at OSSEU 2017 (Recap)

Georg Link glink at unomaha.edu
Tue Oct 24 21:33:54 UTC 2017

Hi CHAOSS Metrics Committee Members,

We had a great time today at OSSEU in Prague. I summarize my thoughts and
takeaways from today for everyone who could not make it and to move the
conversation forward.

The agenda had roughly thee topics:
1) Introduce CHAOSS and discuss onboarding
2) Talk about software implementation experience and issues
3) Diversity and Participation

# 1. Introduce CHAOSS and discuss onboarding

We started by introducing the CHAOSS community and its structure with two
technical committees.

At one point, we talked about the Growth-Maturity-Decline metric [1] and I
recorded the following feedback:
- Metrics are currently developer-centric and should be balanced to
incorporate the software users (without users, no soil for new
contributors) – this could include looking at user forums, user groups,
meetups, and other interactions for mainly users.
- The “Bug Time to Close” metric is good but there are other
‘timeliness’/’responsiveness’ measures, such as “Time to response email”.
Inability to keep up with requests, because too many come in, could
indicate that documentation is poor and some information is not getting to
the right people.

We closed the first agenda item by talking about the information a newcomer
to the CHAOSS community needs and where to position this information. The
notes for this can be viewed and discussed in a pull request:

# 2. Talk about software implementation experience and issues

The second agenda item was to exchange experience and issues concerning
software implementation. We heard from Łukasz Gryglicki about his work on a
CNCF dashboard. Josianne Marsan (Laval University) introduced the
initiative SECOHealth which has the same objective as CHAOSS and is
currently conducting interviews and will feed results back to the CHAOSS
project. Sean Goggins presented work on the GHData project as a way to
prototype the metrics we define in the Metrics Committee. J. Manrique López
introduced the three official CHAOSS projects Cregit, Prospector, and

A point of discussion was how to capture demographic information about
community members. Mike Dolan informed us we have to rely on “opt-in”
information to be compliant with laws everywhere. One approach is to have a
“contributors.json” file (or other formats) in a repository where
contributors update their information via pull-requests. Another approach
is to have a community profile maintained somewhere else, e.g. in
combination with the foundation membership registration.

I’m sure there were more interesting things during this discussion and I
invite everyone to share their notes.

# 3. Diversity and Participation

After the afternoon coffee break, we had a great discussion on diversity,
inclusion, and participation.

Feedback on the current Diversity and Participation metric [2]:
- Diversity is only captured in the metric “Contributor Demographics” – the
other metrics are about participation. Combining the two kinds of metrics
into one group appeared to cause confusion today and we might have to
either communicate better that diversity and participation are highly
interconnected, or separate them and have two groups of metrics.
- The way I understand how diversity and participation relate to each other
is that for groups (e.g. gender: women and men, or ethnic background:
Caucasian and Asian and …) the participation metrics are compared between
groups. If for example participation metric “maintainer promotion” is much
higher for one group than for another, it shows low diversity. The same is
true when “new contributions” is 2% for one group and 98% for the
comparison group(s).
- The metrics do not make it clear that we understand “contributions” to
include technical and non-technical contributions.
- We should include the importance of recruitment and retention as equally
valuable metrics.
- None of our metrics captures “seniority” of diverse groups – how are
different groups represented in leadership positions, managerial and

Daniel Izquierdo presented the structure of the “Gender Diversity Analysis
in the OpenStack Community” report [3] which he co-authored with Nicole
Huesman and Allison Price. The work was well received.
Daniel proposed to continue the work on diversity reports in the CHAOSS
community with the goal to standardize diversity metrics. The diversity
report could serve as a starting point for CHAOSS to create agreeable
diversity definitions, standardize metric definitions including a definite
way to calculate each metric, build tooling in the software committee for
generating the metrics, and finally have a template for the report that
communities can use and fill in their own numbers. The work around the
diversity report can serve as a front-to-back use case that CHAOSS can
start working on and in the process define its processes.

One comment during the meeting was that as a young CHAOSS community, we
lack shared definitions and have to start with the basics. As such, I
propose that we facilitate all discussion within the existing CHAOSS
structures until we achieve a shared understanding and create a need to
separate work. Specifically, I think we can start work on the Diversity
Report use case on the metrics mailing list until we defined the metrics
and then continue the conversation on the software mailing list for
building tooling. This approach helps us all evolve with the discussion. A
concern with this approach is that it will get messy – and I agree – but
the messy discussion is necessary for us to build the CHAOSS community and
define how we see the world as a community. We already have three software
projects that started outside the community and we see how difficult it is
to bring them together and onto the CHAOSS GitHub repository – I would like
to avoid something similar happening with metrics initiatives. Again, my
proposal is to start the Diversity Report use case initiative on the
metrics mailing list, open for all to participate, and with everyone
expecting a messy but productive discussion.

This concludes my notes and thoughts from today. Feel free to cut the email
and respond to only the sections that interest you. Everyone is invited to
respond, share their perception of what we talked about today, and add
thoughts (even if you were not present) on how we should proceed and what
we should build next.



Georg J.P. Link
PhD Student
College of Information Science and Technology | PKI 367
University of Nebraska at Omaha | www.unomaha.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/oss-health-metrics/attachments/20171024/a5bfeae0/attachment.html>

More information about the Oss-health-metrics mailing list