In computer science, is omission of negative experimental evaluation results research misconduct?
tl;dr- In general, suppressing negative results harms objective analysis but can make the research appear more significant. How people feel about this practice depends on their stake in it.
Pure consumers are the most likely to feel that it's misconduct.
Pure investees are the most likely to feel that it's acceptable/appropriate.
Others in the field likely have mixed feelings since they benefit from the practice as investees but are harmed by it as consumers.
Looking forward, there's a growing understanding that these practices pollute the literature and will need to be weeded out. However, we're not quite there yet.
Pure consumers are likely to consider it misconduct
At one extreme, readers who have no affiliation with the research are the most likely to object. Such readers might be trying to select an algorithm for their own research or business application; they want to know the good and the bad equally, so having the bad omitted is purely detrimental to them.
For example:
Ashley has developed algorithm X and decided to experimentally compare it with algorithm Y. She compared them on benchmark set A, B and C. She found that on benchmark C, algorithm Y outperformed X, so she decided to not report on C. In the paper, she also only claims algorithm X outperformed on A and B.
If a reader is trying to select which algorithm to use, then they'd likely want all relevant benchmarks. Doubly so if Benchmark C is more closely related to their application.
Pure investees are most likely to consider it acceptable/appropriate
At the other extreme, those invested in the research effort itself are most likely to want to see it presented in a positive light. Investees include the researcher themself; their supervisor(s); their institution; and any media services that report on their work (e.g., journals).
Pure investees are those who aren't also consumers. For example, a university's promotional news team is a pretty pure investee, as they basically want to make the research shine.
For example:
Ashley has developed algorithm X and decided to experimentally compare it with algorithm Y. She compared them on benchmark set A, B and C. She found that on benchmark C, algorithm Y outperformed X, so she decided to not report on C. In the paper, she also only claims algorithm X outperformed on A and B.
If Ashley's supervisor feels strongly about the result, they may go into full-promotional-mode, including contacting the university's promotional team and others to advertise the work. Ashley's supervisor and related promoters might wench at any criticism or negative analysis that might detract from their efforts, so they're more likely to appreciate the comparison using Benchmark C not being reported.
Fellow practitioners may have mixed feelings
Other computer scientists may have mixed feelings since they're consumers, but likely engaging in the same behaviors.
In practice, I've seen researchers acknowledge that they understand such behaviors to be detrimental, but still argue that not downplaying/omitting criticism would make their work appear unduly weak compared to other researchers'.
Overall, practitioners seem to generally understand that it's sort-of misconduct in the sense that it shouldn't be done, but that it's acceptable in the sense that authors often feel like they must do it to play on a level playing field with others who do.