[Chaoss-software] [Meeting item] Collaboration of projects within the Software TC

Jesus M. Gonzalez-Barahona jgb at bitergia.com
Sun Oct 22 21:48:36 UTC 2017


On Sat, 2017-10-21 at 16:44 -0700, D M German wrote:
>  >> so in my opinion, and maybe i am naive Jesus, all you have to do
> is
>  >> point your tools to the token view repo as if it was another
>  >> repository
>  >> and nothing else.
>  >> 
>  >> I would also be curious if people think that there are some views
>  >> that
>  >> they would like to see.
>  >> 
>  >> Currently cregit supports: C, C++, Java, go and python.  It takes
> a
>  >> bit
>  >> of work to get it installed, but it works. If it can process
> Linux it
>  >> can process any repo :) 
> 
>  Jesus> Daniel, if you produce a GrimoireLab dashboard for a git
> repository
>  Jesus> produced with cregit, you get exactly the same than for a the
> original
>  Jesus> repository. Because it just work at the level of commits, and
> as you
>  Jesus> know, commits, including files touched, people authoring the
> commit,
>  Jesus> etc. is exactly the same.
> 
> Hi Jesus,
> 
> I think we need to separate metrics computation from matrics
> displaying. My suggestion is that we should have a clear layer that
> separates visualizations from metrics extraction.
> 
> And maybe what I need is to understand each of the tools you are
> developing.
> 
> GrimoireLab displays metrics, but I suspect also computes them.
> 
> So it really have two layers: extraction/visualization.
> 
> 1. What is the data model you use?
> 
> 2. How do you store the information? (i am affraid the answer is
> elastic
>   search :)
> 
> 3. Why are they not two different tools (split visualization from
> metric
>    extraction)?
> 
> if you separate the layers we can work independently on creating the
> metrics, and you can work on visualizing them.

Yes, I think we need some clarification here. Sorry for not being more
clear earlier. The basics of an answer to your question are at:

https://grimoirelab.gitbooks.io/training/grimoirelab/intro/components.html

and 

https://grimoirelab.gitbooks.io/training/grimoirelab/intro/scenarios.html

In short, we have a level, which is Perceval, which just extracts
information from the APIs and produces JSON documents. Those are
available directly from Perceval, as Python dictionaries, or from an
ElasticSearch database, as JSON documents (that's what we call "raw
indexes"). Other databases are perfectly possible, and easy to
implement: just upload the documents to them. This layer tries to mimic
as much as possible the original data source, so from a semantic point
of view is not a "layer": it just provides more convenient availability
of data.

Then, GrimoireELK produces enriched indexes. Those may include actually
some computation of metrics, such as time to close a ticket. Those are
again JSON documents, and again are stored in ElasticSearch, although
they could be uploaded somewhere else, and in this case even easier,
since they are just flat JSON documents.

Then, metrics can be computed from these indexes in several ways. We
mainly use two: reports and Kibana. The first one computes specific
metrics, producing CSV data (ready to be consumed, or included to
produce PDF documents). The second one computes metrics usually in the
browser (like counting commits), because there is no other way of doing
that and allowing for advanced filtering and drilling down.

Software for computing and/or visualizing metrics can exploit data at
any of these levels. For example Prospector works directly on top of
the Perceval API, and could easily work on top of raw indexes.

In the end, GrimoireLab is not intended to produce an specific set of
metrics. It is designed to provide services at different levels for
production and visualization of many different kinds of metrics. That's
why I think it should be relatively easy to tailor it to produce at
least some of the metrics defined by the Metrics TC.

This said, I know the GrimoireLab model is far from perfect, and that
it needs clarification and probably redesign from many points of view.
I'm happy to have a discussion in that direction.
  
Saludos,

	Jesus.

> --dmg
> 
> 
> --
> Daniel M. German                  "Operating systems are like
> underwear,
>    Bill Joy ->                     nobody really wants to look at
> them."
> http://turingmachine.org/
> http://silvernegative.com/
> dmg (at) uvic (dot) ca
> replace (at) with @ and (dot) with .
> 
-- 
Bitergia: http://bitergia.com
/me at Twitter: https://twitter.com/jgbarah



More information about the Chaoss-software mailing list