python: a little bit about dunder functions

What I learned from looking at Fluent Python yesterday:

• Functions defined with surrounding double-underlines (“dunder”) are called differently
•  e.g. len(my_object) instead of my_object.len()
• In general, we shouldn’t create new dunder  functions
• One reason is that newer versions of Python might use the same name
• __repre__ is called by __str__ if no __str__ defined, so it’s more important to define __repre__()  
• If no __bool__ defined, then python uses __len__() and returns False if 0, True otherwise

Advice on Managing SW Enhancement Requests - Links

From https://www.novell.com/communities/coolsolutions/what-happens-enhancement-requests/:

Once I have seen the request I generally do one of 4 things with it almost immediately.
1. If I know that this is a duplicate of another request I close the new one and mark it as a duplicate of the original.
2. If I see no value to the request, or I see value but the opportunity cost is too high, then I will just close it.
3. If I see value to the request, and it fits in with much of the feedback received on all of the other channels then I will assign it to be considered for a release
4. Finally I can choose to do nothing with it right now. This final one allows me to bear it in mind and use it for consideration on future requests.

From http://support.aha.io/hc/en-us/articles/202000757-Best-practices-to-managing-and-prioritizing-features:

1. You need to know what the goals or key business drivers are for your product so you can create a scorecard that includes these key metrics.;
2. If it is not clear who owns prioritization, your scoring will not be trusted. It's essential to know who is in charge;
3. If there is not enough detail per feature, then the functionality of each feature will be unclear.

From http://johnpeltier.com/blog/2013/12/12/feature-requests/:

1.  Ideas that don’t fit with the long term vision for the product at all.  These receive a polite rejection.
2. Many, many good ideas that you might do someday.  These are usually acknowledged but sit in the “someday” bucket.
3. A few ideas which map to the area of the product that needs work next, or the next most important problem to solve.

Once prioritized, you’ll want to communicate your plans in a roadmap that doesn’t tie you to specific dates.

From https://www.quora.com/How-should-a-product-manager-handle-communication-to-internal-people-who-make-feature-requests-when-the-volume-of-requests-far-exceeds-engineering-capacity:

The backlog should not be a dumping ground for all ideas that may get worked on eventually, or may not. It will become impossible to manage and just gives the requestor a false sense of security by having it added. If you don't see yourself working on a backlog item in the next ~4 months, I recommend just telling the person no, point blank, or maintaining a separate list or roadmap outside of the backlog for these items.

python: initialize a dictionary with input arguments key value key value etc

If you invoke scriptname with list of key names followed by their values.
e.g.
         python scriptname.py A 1 C 3 B 2

Then you could parse it like this:

dd={}
for i in range(1,len(argv),2):
    dd[sys.argv[i]]=sys.argv[i+1]

Toy code to test:

>>> params=['scriptname','A',1,'C',3,'B',2]
>>> dd={}
>>> for i in range(1,len(params),2):
...     dd[params[i]]=params[i+1]
...   
>>> dd
{'A': 1, 'C': 3, 'B': 2}

python: supply default values for incomplete dictionary

>>> default_dd={'A':1,'B':2,'C':3,'D':4,'E':5}
>>> dd={'B':2000,'D':4000}
>>> for key in default_dd.keys():
...     if not key in dd.keys():
...         dd[key]=default_dd[key]
...       
>>> dd
{'A': 1, 'C': 3, 'B': 2000, 'E': 5, 'D': 4000}
   
   

python: initialize a dictionary with uncertain number of input values

If you know the order of the expected input parameters, but you don't know how many will be supplied:


if __name__ == "__main__":
    #initialize list of expected parameters, in order
    dd_parameters=['scriptname','A','B','C','D','E']
    dd={} 
    if len(sys.argv) > 1: 
        #loop through the actual input parameters
        for i in range(1, len(sys.argv)):
            dd[dd_parameters[i]]=sys.argv[i]

See, you automatically stop at the end of the number of input parameters.
Here's some toy code to test the logic:

 >>> std_parms=['scriptname','A','B','C','D']
 >>> input_parms=['scriptname','A_Value','B_Value']
 >>> dd={}
 >>> for i in range(1,len(input_parms)):
 ...     dd[std_parms[i]]=input_parms[i]
 ...
 >>> dd
 {'A': 'A_Value', 'B': 'B_Value'} 

After that, you may want to fill in the default values for the parameters not supplied.

Quotes and Notes from 'Becoming a Better Programmer' - Chapters 9-10

From O'Reilly e-book:
Becoming a Better Programmer by Pete Goodliffe

Chapter 9 - Expect the Unexpected

Any function you call may not work as you expect.
• If you are lucky, it will return an error code to signal this. (p.69)

p.70:

•  understand basic concurrency principles
• Understand mechanisms to reliably and quickly pass messages between thread contexts without introducing race conditions or blocking the threads unnecessarily
•  Shutdown:
• how to bring the code to a graceful halt without leaking resources, locking up, or crashing
• especially hard in a multithreaded system
• don't leave one object attempting to use or return to another that has been deleted

Chapter 10 - Bug Hunting

• Lay Traps: Add assertions or tests to verify the system invariants—the facts that must hold for the state to be correct. (p.78)
• Binary Chop: (p.78)
  
• Work out the start of a chain of events.
• Work out the end
• Check middle point
• if good, then start to middle is good, so check midpoint of here to end
• if bad, then check midpoint of start to this point
• Software Archaeology
• Look thru your version control to find the last time this bug didn't exist
• Binary chop through time to find where bug was introduced
• Use a code coverage tool to inspect how much of your code is actually covered by tests. (p.79)
• Once you find the bug:
• Write a simple test that demonstrates the bug.
• Add it to the test suite before you fix the code.
• Use the test to:
• prove bug is fixed
• test to prevent bug from reoccurring
•  Analyze/look for related bugs
• Are there more bugs in the section where you found this one?
• Was bug in a repeated code pattern that's used elsewhere?
• Note in which parts of the code you're finding more bugs.
• Part of code that's used most often?
• Weakness in that particular section of the software? 
• Non-Reproducible Bugs
• Keep records that may eventually help you come up with theories you can test to possibly find the cause.
• Add more assertions and log statements -- preferably into your beta or release builds.
• Automate a test farm to run long-running soak tests.
• Common situations leading to hard-to-reproduce bugs:
• Threads
• "Heisenbugs" - disappear when you add log statements
• Network interacation
• Speed to access database, disk, other storage can vary
• Memory corruption
• Global variables

Quotes and Notes from 'Becoming a Better Programmer' - Chapters 2-8

From O'Reilly e-book:
Becoming a Better Programmer by Pete Goodliffe

Chapter 2

• KEY ➤ Remember who you’re writing code for: other people. (p.9)
• page 10:

Some programmers like to present banner comments in pretty ASCII-art boxes:
/**************************************************
* This is a pretty comment. *
* Note that there are asterisks on the *
* righthand side of the box. Wow; it looks neat. *
* Hope I never have to fix this tiypo. *
**************************************************/

...Frankly, this is a sadistic presentation style.

• Also on p.10, he says to write code as you would prose. "Functions are akin to chapters."
• Within the chapters, related lines of code are equivalent to paragraphs.
• Insert blank lines only between these conceptual paragraphs, not within them
• Don't have too many of these "paragraphs" per function

• If you clean up code, make that a separate commit from any changes to the code's behavior.
  KEY ➤ Never alter presentation and behaviour at the same time. Make
  them separate version-controlled changes. (p.14)

Chapter 3

• (p.18) There are many reasons why unnecessary code is the root of all evil...
• Writing a fresh line of code is the birth of a little life form.
• ...
• Code harbours bugs. The more code you have, the more places there are for bugs
  to hide.
• Refactoring (p.20) 
• A refactor is only a refactor if it is a transformation of the source code, preserving behaviour.
•  the DRY principle: Don’t Repeat Yourself!
• ...factoring similar code into a shared function introduces tight coupling...
  They both now rely on a shared interface;...
  —so DRY your code responsibly! (p.21)
• Dead code is code that is never run, that can never be reached...
• Tell your code to get a life, or get lost. (p.23)
• The code itself says what and how. A comment should explain why..(p.24)
• Do not remove code by commenting it out. (p.24)
• Remove code completely. 
• You can always get it back afterwards from your source control
  system.

Chapter 4

• YAGNI = You Aren’t Gonna Need It: (p.29)
• Don't write..code you think is going to be needed in future versions. 
• Don’t write it now if you don’t need it now. 
• Much "debug" code is necrotic...unsightly scaffolding that hides the beautiful architecture underneath. (p.31)
• It is harder to refactor, simplify, or optimise your program when it is bogged down
  by zombie code. (p.32)
• Don’t add “minor”features that you think are interesting, but no one has asked for. They’ll be easy enough to add later, if they are required. Even if it seems like a good idea. Don’t do it. (p.33)  That one is hard to resist. :^)

Chapter 6

• ...try adding a single, one-line, failing test. Does the test suite immediately fail? This smoke test proves that the tests are not actively being ignored. (p.50)
  

Chapter 7

• I know I shouldn’t moan, but sometimes I swear that programmers shouldn’t be allowed
  to type the word thread without first obtaining a license to wield such a dangerous
  weapon. (p.56)
• ...if it has been working adequately for years without tinkering, it is probably inappropriate to “tidy it up” now, especially if you’re unlikely to need to make many more changes in the future. (p.58) 
• KEY ➤ Make code changes slowly, and carefully. Make one change at a time. (p.58)
• ...the unreadable mess of special cases might be like that for a reason. Each bodge and code hack encodes an important piece of functionality that has been uncovered through bitter
  experience. Ignore these subtle behaviours at your peril. (p.59)
• The effective programmer does not only deal well with the bad code, but also with the people that wrote it. (p.60)
• “refactored the code to look neater.” -> “refactored code back to working.” (p.60)

Chapter 8: 

• Don't ignore:
• Return codes -  e.g. printf
• Side effects - e.g. errno - which is a mess when code is threaded
• Exceptions

try
{
    // ...do something...
}
catch (...) {} // ignore errors
The saving grace of this awful construct is that it highlights the fact you’re doing
something morally dubious. (p.64)

• Understand your language’s idioms and requirements for effective exception use. (p.65)

Using Workflowy to dissect a task into do-able bits

When you're finding yourself having trouble getting started with something on your to-do list, start subcategories under the main task, and include in it all the questions and obstacles you see, and then the sub-task questions and obstacles and tasks that exist under them. Keep creating the sub-tasks until you see the concrete things that you can do first. 

Create free lists like this at  https://workflowy.com/

python: Nonintuitive ELSE after FOR or WHILE

When I first saw an else clause after a for loop, I concluded that those were the steps to run if the condition inside the loop was never true-- that is, either you enter the loop or you do its else.
Wrong: Those are the steps to run if you exit the loop without a break statement. 

A better label would be NO_BREAK, but I understand not wanting to add keywords to the language. 

Test: Are the else steps executed when the loop is never entered?

for thing in ['abc','','tobreak','ok']:
    print "\nvertical [%s]" % thing
    for c in thing:
        print c
        if c=='r':
            break
    else:
        print "No r in [%s]" % thing
else:
    print "\nthat's all"

output

vertical [abc]
a
b
c
No r in [abc]

vertical []
No r in []

vertical [tobreak]
t
o
b
r

vertical [ok]
o
k
No r in [ok]

that's all

Process finished with exit code 0


SVN - unversioned contents in SVN-renamed directory

I noticed this using Tortoise SVN in Windows:

If a directory is renamed under SVN and you then update its parent directory, your current directory will be deleted, and a new version of the directory, with the new name, and all of its contents, will be downloaded back to your checked-out version of the parent directory.

This can be a real pain when there are a lot of big files that are deleted and then downloaded back again. Enough to rant about, and want a better way of implementing SVN Rename, but that's not the subject of this blog entry. The subject of this blog entry is:

If you have unversioned items in a directory that is in SVN, and you (or someone else) changes the name of that directory in SVN, then after the commit or update, SVN will leave a copy of the directory with the old name, containing the unversioned contents.

That is:

1. I have a directory: Dir-v, which contains versioned file File-v and unversioned file File-u.
2. Someone changes the directory name to newDir-v and commits the change.
3. I update Dir-v's parent directory.
4. I now have two directories: 
1. versioned newDir-v containing File-v 
• in Tortoise/Windows you'll see a green check mark on their icons
2. unversioned Dir-v containing unversioned File-u
•  in Tortoise/Windows, you'll see a blue question mark on their icons
5. I can delete the unversioned Dir-v after deciding what I want to do about File-u.

Why do I have unversioned files in my SVN directories? Well, there are some local files that I want to be categorized with, and able to find when I'm looking at other files that are in the SVN, but I don't think the other people in the project want these files in their copy of the structure. Is that bad?