[cheesecake-dev] Negative scores?

[Will Guaraldi]

On Sat, 3 Feb 2007, Grig Gheorghiu wrote:
> 
> On 2/3/07, Micha? Kwiatkowski <constant.beta at gmail.com> wrote:
>>
>> Second solution is to make pep8 compatibility score always greater than 
>> zero, with the highest score for packages with 0 errors and 0 warnings. 
>> This could be achieved by counting all possible errors and warnings of 
>> pep8 module and calculating a maximum value from that. This would 
>> remove negative scores while still representing changes in scores.
>
> I think the second solution is much better. I'm CC-ing Johann, the 
> author of pep8.py, because he was saying at some point that he might 
> introduce a scoring method in pep8.py itself. That would make things 
> even easier for consumers of pep8.py such as Cheesecake. But in the mean 
> time, I think your second solution is really good, Michal.

Mmmm....  I think the second solution is the best of the two as well, but 
with a metric that has a potentially infinite penalty, I'm not really sure 
how Johann can implement pep8 scoring.  I think it'd have to be a ratio of 
some kind of errors/error-types to large-ness of the code so that you 
could get a "blah out of 100" scoring that's relevent--or something like 
that.

Regardless, I'm not wildly excited about the pep8 scoring in the first 
place.  The top of pep8 talks about how it's better to have readability 
than a strict conformity of pep8.  On top of that, while pep8 does provide 
a decent style guide and they use it for the stdlib, I don't think that 
it's the _only_ good style guide.  I think it's better for cheesecake to 
test for style consistency within a project rather than strict conformity 
with pep8.

I think that questions like "are there unit tests for this project" are 
important, but questions like "does this project use 4-space indentation" 
are not.  Doing a skim of pep8 module, I think many of the pep8 rules fall 
into the latter category.

I think this particular check hurts projects more than it helps people as 
it's currently implemented.  I think in order to get around the issues I 
have, projects would need to be able to specify the coding style to check 
for consistency on.  Perhaps some specification method like they have for 
checkstyle (http://checkstyle.sourceforge.net/).  Something like that with 
a very lenient default would be better.

On a side note, I think of all the things that cheesecake checks for, 
checking style is going to be up there in the complaints department.  You 
might as well tell everyone that their bikeshed has to be blue or no 
points!  ;)

Hope that all makes some modicum of sense--

/will