Are the social smells implemented in Kaiaulu the same as Codeface4Smells?

[carlosparadis]

@tuejari I'm sharing your question here since this may benefit the others also using the metrics.

The notebook mentions that “Kaiaulu will then perform a non-overlapping time window of every 3 months of git log history and mailing list to identify the number of organizational silos, missing link and radio silence social smells." In the output I notice a column for primma_donna (although the output is 0). I was told that Kaiaulu essentially detects community smells in the same way they were detected using CodeFace4Smells. In the showcase and in the original paper proposing CodeFace4Smells (https://ieeexplore.ieee.org/document/8651329), the primma donna smell is not mentioned.

The paper you cite uses both data and metrics from Simone's dissertation (https://www.politesi.polimi.it/bitstream/10589/123826/3/2016_07_Magnoni.pdf),
but some metrics were renamed (see the social smell chapter in Simone's dissertation). The code from Codeface4Smells that does the social smell metrics is:
https://github.com/sailuh/kaiaulu/blob/88-add-social-smells/sociotechnical.R.

In it, you will find both the quality framework metrics and the social smells. The social smells in particular were defined as functions, e.g. primma donna:

kaiaulu/sociotechnical.R

Lines 204 to 205 in 49d6a55

	community.smell.primadonnas <- function (mail.graph, clusters, code.graph, collaboration=0.2,
	precomputed.black=NA) {

, which is also defined in the
dissertation. In fact, I'd recommend you use the dissertation as source material also to understand instead of the paper: The dissertation includes
one additional diagram to the paper (since it has no space limitation), that makes it easier to understand the underlying implementation to the theory.

If you compare the implementation of social smells of the code above to https://github.com/sailuh/kaiaulu/blob/88-add-social-smells/R/smells.R you will
see they are the exact same (save perhaps for me renaming the function name from dots to underline (_).

This is what is meant by them being the same, but that does not mean I re-implemented everything: Most of them are scattered in the original source
code (https://github.com/sailuh/kaiaulu/blob/88-add-social-smells/sociotechnical.R), and I did not get around yet "fishing them all".

Another thing you should consider is whether what goes as input to the functions you see, i.e. the git log network (code.graph), and the mailing list network (mail.graph), along with their respective "clustering. So in this Figure which shows the possible ways you could create the social smells notebook (which is one path of it):

You will see the code graph and mail graph could be derived in two ways (temporal or bipartite). The code graph could also be done at file or function granularity. I am not certain which path the TSE paper or Simone's dissertation took. The community detection algorithm is the same, however, which is random walk. At least on the social smells branch.

Lastly, @maelstromdat and @fpalomba had some concerns with the Codeface4Smells in the dissertation, so you may want to check with them before using this metric (I don't recall what specifically was the issue).

In the diagram above, it also states that primma donna is implemented. So, are the community smells detectable by Kaiaulu limited to organizational silos, missing link, and the radio silence smell (as reported in the showcase)? Or does it also support the primma donna smell?

Primma donna is implemented, although there are the concerns above. A few quality framework metrics are also implemented, which I believe both TSE paper and dissertation show:

kaiaulu/R/smells.R

Lines 29 to 37 in 49d6a55

	community_metric_sociotechnical_congruence <- function (mail.graph, code.graph) {
	missing.collaborations <- length(community_smell_missing_links(mail.graph, code.graph))
	healthy.collaborations <- length(E(code.graph)) - missing.collaborations
	congruence <- healthy.collaborations / (length(E(code.graph)))
	if (is.na(congruence)) {
	congruence <- 1
	}
	return(congruence)
	}

kaiaulu/R/smells.R

Lines 78 to 105 in 49d6a55

	community_metric_mean_communicability <- function (mail.graph, code.graph) {
	## If no collaborations, communicability is 1
	if (length(E(code.graph)) == 0) {
	return(1)
	}
	## for each collaboration compute its in-communicability value
	collaborations <- get.edgelist(code.graph)
	mai <- c()
	for (coll in 1:length(E(code.graph))) {
	id.dev1 <- V(code.graph)[V(code.graph)$name == collaborations[coll, 1]]$name
	id.dev2 <- V(code.graph)[V(code.graph)$name == collaborations[coll, 2]]$name
	neigh1 <- neighbors(code.graph, V(code.graph)[V(code.graph)$name == id.dev1])$name
	neigh2 <- neighbors(code.graph, V(code.graph)[V(code.graph)$name == id.dev2])$name
	code.neighbors <- unique(union(neigh1, neigh2))
	dem <- length(code.neighbors) / length(V(code.graph))
	neigh1.mail <- c()
	neigh2.mail <- c()
	if (id.dev1 %in% V(mail.graph)$name) {
	neigh1.mail <- neighbors(mail.graph, V(mail.graph)[V(mail.graph)$name == id.dev1])$name
	}
	if (id.dev2 %in% V(mail.graph)$name) {
	neigh2.mail <- neighbors(mail.graph, V(mail.graph)[V(mail.graph)$name == id.dev2])$name
	}
	mail.neighbors <- intersect(unique(union(neigh1.mail, neigh2.mail)), code.neighbors)
	dam <- length(mail.neighbors) / length(V(code.graph))
	mai[length(mai) + 1] <- dem - dam
	}

This is because these are defined as actual functions in the original code.

There is also a few more that will show up in the final table from the quality framework which I implemented based on the conceptual definition (i.e. number of timezones, number of authors on git log, number of authors on the mailing list, etc).

I assume that the Black Cloud Effect smell is not yet supported (as depicted in the diagram), is that correct?

Yes, black cloud is not implemented. Although the potential black cloud function is:

kaiaulu/R/smells.R

Line 277 in 49d6a55

community_smell_potential_black_cloud <- function (mail.graph, clusters) {

so implementing it shouldn't take more than a for loop. Note in the original code:

kaiaulu/sociotechnical.R

Lines 722 to 726 in 49d6a55

	smell.black.cloud <- intersect(smell.potential.black.cloud, prev.range.potential.black.cloud)
	prev.range.potential.black.cloud <- smell.potential.black.cloud
	all.smell.blackcloud[range] <- length(smell.black.cloud)
	## Reuse black cloud computation to compute prima-donnas. Prima-donnas can exist only when
	## a black cloud exists

I did not finish its implementation, because this metric requires a comparison of current and immediately before time windows after community detection. Org silo, missing link, radio silence and primma donna are single time slice metrics. Also the original results of the metric seemed at a quickly glance to give mostly 0s. You can find them here:

https://github.com/maelstromdat/codeface4smells_TR/tree/master/Results (see any project folder/results.csv), e.g.:

https://github.com/maelstromdat/codeface4smells_TR/blob/master/Results/Capistrano/report.csv

Note all of the above only applies to the social-smells-branch: The social smells in the master branch (which I only kept radio silence, missing link and org silo) are refactored to not rely on igraph, and the community detection algorithm is different (arguably more robust). It was not ever clear to me why Simone dissertation and the associated social smell implementation opted in for the igraph's random walk. Codeface and associated papers itself use OSLOM instead (which is what the master branch uses). See the NEWS.md of release 0.0.0.9600 for details.

[carlosparadis]

Accidentally replied in the question post, and Discussions is not letting me edit for some reason 😅 if it ever does I will edit these two.

[tuejari]

Thank you for the clear and elaborate answer!

Lastly, @maelstromdat and @fpalomba had some concerns with the Codeface4Smells in the dissertation, so you may want to check with them before using this metric (I don't recall what specifically was the issue).
Primma donna is implemented, although there are the concerns above.

I have re-read the part from Magnoni's paper about the Prima-donna Smell and found the following concern (just to add to the discussion for future questions):

If computed collaboration level is over a given threshold, then a Prima-donnas Effect Community Smell is detected. The threshold used to decide if two isolationist sub-communities are collaborating, thus if they should be considered as Prima-donnas, is not universally definable because it may depend on the software development context, characteristics and research
preferences.

Org silo, missing link, radio silence and primma donna are single time slice metrics

kaiaulu/sociotechnical.R

Lines 725 to 726 in 49d6a55

	## Reuse black cloud computation to compute prima-donnas. Prima-donnas can exist only when
	## a black cloud exists

Furthermore, I found this comment confusing. It seemed contradictory to say that the Prima-donna smell is a single time slice metric, while also in the comment stating that the smell can only exist when a black cloud exists. Magnoni's dissertation states that step one of identifying the prima-donna smell is:

identification of sub-communities that communicate with eachother exclusively through a unique communication link, thus it is the same concept the concept of “potential” Black-cloud Effect Community Smell;

So to me it seems that the comment should read: 'Prima-donnas can exist only when a "potential" black cloud exists'. I do realize that you mentioned that the code in https://github.com/sailuh/kaiaulu/blob/88-add-social-smells/sociotechnical.R is code from CodeFace4Smells, but I just wanted to clarify it in this discussion.

[carlosparadis]

Yes, you are correct. Note in the actual function this is worded properly:

kaiaulu/sociotechnical.R

Lines 209 to 210 in 49d6a55

	## For every potential black-cloud, check the collaboration of the involved sub-communities.
	## if it is greater than the threshold, we have two prima-donnas

Also, just to be clear Kaiaulu does not use this file directly, but "smells.R" (which has the exact implementation from the function code in the social-smells branch). The master branch re-implements them to not depend on the graph library, and to use OSLOM community detection instead of igraph's random walk. I also decided not to port over primma donna nor black cloud for now to the master branch.

[tuejari]

Yes, you are correct. Note in the actual function this is worded properly:

Apologies, I hadn't seen that it was worded properly in the actual function. I made the remark based on your initial answer.

Thank you again for your help!