Linux: basename

basename is cool.
Takes out any piece of a filename you'd like to get rid of.  It doesn't take wildcards though.

$ basename ABC_s.vpf _s.vpf
ABC

$ basename AB123 3
AB12

$ basename AB12_34 12*4
AB12_34

It also removes the directory names from the path:

$ basename a/b/c/d/AB12_34
AB12_34
$ basename a/b/c/d/AB12_34 2_34
AB1

Linux Link

head, tail, cat, tac, nl, fmt, fold, tr, pr, basename, dirname, all described as Dogs of the Linux World.

Linux: EBCDIC to ASCII conversion

To convert from EBCDIC to ASCII:

$ iconv -f EBCDIC-US -t ASCII  EBCDICinput > ASCIIoutput

You can also use dd:

$ dd if=EBCDICinput of=ASCIIoutput conv=ascii

$ file -i EBCDICinput 
EBCDICinput : text/plain; charset=iso-8859-1
$ file ASCIIoutput
ASCIIoutput: ASCII text, with very long lines, with no line terminators

Though I've found the first one (iconv) will keep pipe characters as "|", and the second (dd) will have them as "!" in the ASCII output file.

Windows Explorer XP 5.1 Windows often hangs when I open a file

The issue is that my windows explorer (what a Mac user would call the Finder Window) keeps hanging on me.  In the folder-name bar, it has the folder name, then "(Not Responding)". I end up clicking the red X at the top right to close it, then selecting 'End Program', letting it start up again, and then opening a new windows explorer and navigating my way back to the folder.  This happens often when I double click to open a file.  It happens frequently and is getting annoying.

Someone suggested that I defragment my hard drive.  I've run the defragger over and over until there's not much left to de-frag.  No help.

Search on Microsoft site didn't help.
Attempt to do a Microsoft updatate didn't show any patches for XP 5.1, service pack 3.

Google search resulted in http://www.analogduck.com/main/explorer_hangs  which links to http://www.annoyances.org/exec/forum/winxp/t1167888467

I followed the advice from http://www.annoyances.org/exec/forum/winxp/t1167888467 here:

To stop ctfmon from autostarting and running all the time:
Control Panel>Regional and Language Options>Languages>Details>Advanced,
 and check the box that says "Turn off advanced text services." 

And I've gone into the Task Manager and stopped ctfmon.exe.

I've gone into the Control Panel/Administrative Tools/Services window and stopped the SSDP Discovery Services (along with other services that I didn't think I needed.)

Let's see if it helps...

2:56pm.  The (Not Responding) happened when I tried to open a text document by double-clicking, but it didn't last as long.
7pm.  No, I'm still getting the hanging when I try to double-click to edit a text file.  And I have re-booted since the previous changes.
Oh well.

python day 2: functions, if, else, TypeError

To create a function, define it before you need to use it, and don't forget the colons. Not for the if either. (Python seems to like colons. It's kind of like how C likes semicolons, and Lisp likes parentheses.)

import os, sys

def run_cmd(cmd,fd):
   try:
       if sys.platform.startswith("win")
           print cmd
           fd.write('running cmd from win: '+cmd+'\n')
           EXITCODE = os.system(cmd) >> 8
           #Here's where I got the TypeError:fd.write('cmd returned '+EXITCODE+'\n')
       else:
           import popen2
           process = popen2.Popen3(cmd)
           EXITCODE = process.wait() >> 8
       return EXITCODE
   except:
       print "run_cmd: Some Exception Caught."
       fd.write('run_cmd: Some Exception Caught\n')

The TypeError complained:

    TypeError: cannot concatenate 'str' and 'int' objects

I didn't realize Python was so fussy about types. I thought it was a cavalier sort of Javascript kind of interpreted thingy. OK, so to solve that...it looked like I needed to use a sort of C syntax with the print statements, but that wasn't quite right. I tried it, but got another TypeError:

      fd.write('cmd returned %d\n',EXITCODE)
    TypeError: function takes exactly 1 argument (2 given)

Oh, wow, this really is fussy. Experimenting with the shell again:

    >>> num=3
    >>> num
    3
    >>> print "%d" %(3)
    3
    >>> print "num=%d" %(num)
    num=3
    >>> print "num=%d" % num
    num=3

OK, so it's not C/Java. I need to say:

    fd.write('cmd returned %d\n' % EXITCODE)

Now it works.

One weird thing is that the tutorial I've been using has both %d and %i listed as "signed decimal integer". The python.org page on string formats only lists the 'd' as an integer type, so I'll stick with that. Researching further, there's a section on String Formatting Operations that lists both d and i again. Must be some historic reason for it...

starting python: file I/O, strip(), [start:end]

Janak left me with a python script to process a directory full of files. He said I'd have to modify it a little. He went home.

First thing to look up: To add a comment, put a "#" at the start of the line.
Useful for note-taking in the code, and commenting out the complex stuff while I try to get the easy stuff working.

Well, I have a file I'm writing to:

fd=open("list.txt","w+")
fd.write(var+' is the value of var.\n')

And a file I'm reading from :

for line in open(from_dir + "\\" +in_file)

That is cool to have the read-line from the file in the same step as the opening of the file, and all in a for loop while the lines in the file don't run out, all in one step. And what that syntax does is obvious even to a novice.

I'm not sure about the "\\". This is in Windows, but why two backslashes? Is the first \ to escape the second one? In Linux, would it be "\/" or "/"? Is there a unix/windows-neutral version?

Next, he keeps using this strip() function

line=line.strip()

Here's the official syntax: http://docs.python.org/library/string.html
In a tutorial: http://www.tutorialspoint.com/python/string_strip.htm
If you don't put anything in the brackets, it strips the whitespace from the start and end of the string.

He also uses square brackets to pull out substrings from our lines of data.
The negative value in the range is hard to figure out, but it means "start counting from the end".
If you get a python prompt, you can experiment with square bracket syntax.
It's good to see that it doesn't freak if you try to pull out an impossible range. It just returns an empty string.

>>> s1="abcde.fgh"
>>> s1
'abcde.fgh'
>>> s1[0:2]
'ab'
>>> s1[0:-4]
'abcde'
>>> s1[2:3]
'c'
>>> s1[2:4]
'cd'
>>> s1[-4:-2]
'.f'
>>> s1[-2:-4]
''
>>> s1[5:-3]
'.'

If you're testing the start or end of a string, don't use ranges.
Use startswith and endswith instead:

if str.endswith(".txt"):
if str.startswith("my_files_"):

You'll avoid goofs from miscounting characters, and it's also easier to read and understand.

selective extraction from tar files using pax

I put all the changes I had to make into a single tar file. The tar program removed the leading '/'s from two of the files. I wanted to extract them without creating an entire directory structure. I found this:

http://www.unix.com/unix-dummies-questions-answers/15182-extract-tar-files-without-creating-directory.html

which led me to pax.

I found I could do what I wanted here with three options:

pax -r -i -f [archive] [pattern_matching_desired_file]

The -r is for read from an archive.
The -f is to give it the name of the archive file.
The -i is for name the pulled out file interactively. This meant I had to rename the file, which allowed me to leave out the parent directories.

(There are many more options on the man page, of course!)


Here's how to pull a file out of a tar without having to create all the subdirectories:

$ls
mychanges.tar
$ tar -tvf mychanges.tar
-rw-rw-r-- sam/zac     193 2009-05-18 06:02:23 dir1/changed1.c
-rw-r--r-- sam/zac    3708 2009-05-18 05:57:05 dir2/changed2.py
-rwxr-x--x sam/zac    9708 2009-05-18 05:51:34 dir3/dir4/dir5/dir6/changed3.ksh
-rw-rw-r-- sam/zac    2282 2009-05-18 05:54:57 dir3/dir4/dir5/dir6/dir7/dir8/dir9/cpx.cfg

To pull out the changed3.ksh without creating its parent directories:

$ pax -r -i -f mychanges.tar *.ksh

ATTENTION: pax interactive file rename operation.
-rwxr-x--x May 18 05:51 dir3/dir4/dir5/dir6/changed3.ksh
Input new name, or a "." to keep the old name, or a "return" to skip this file.
Input > changed3here.ksh
Processing continues, name changed to: changed3here.ksh

Now you have your file in the current directory.

$ ls
changed3here.ksh  mychanges.tar

To pull out another:

$pax -r -i -f mychanges.tar *cpx.cfg

ATTENTION: pax interactive file rename operation.
-rw-rw-r-- May 18 05:54 dir3/dir4/dir5/dir6/dir7/dir8/dir9/cpx.cfg
Input new name, or a "." to keep the old name, or a "return" to skip this file.
Input > cpx.cfg
Processing continues, name changed to: cpx.cfg

And there it is:

$ ls
cpx.cfg  changed3here.ksh  mychanges.tar

My Linux/Unix bookmarks: cut, join, sort

cut assumes tab delimiters. To cut using space as a delimiter, use -d' '

$ cat > foo
a b c d
e f g h
h i j k
a b c d
$ cut -d' ' -f 1,3 foo
a c
e g
h j
a c

join join two files by a common field. (after you sort.)

join [options] file1 file2

sort

Installing Firefox 2.0 for Mac OSX.3.9 (so you can run Google docs)

If you have a creaky old mac that can't handle an O.S. greater than 10.3.9, then this is how to find the latest Firefox that runs on it:

You can follow this link

 http://support.mozilla.com/en-US/kb/Installing+a+previous+version+of+Firefox

to the FTP site

ftp://ftp.mozilla.org/pub/mozilla.org/firefox/releases/2.0.0.20/mac/en-US

From there you can download the .dmg file.  Double click on it to install the latest Firefox that will run on X.3.9.

Then you can run Google docs on your old iMac.  Google Docs needs Firefox 2 or Safari 3 to run on X.3.9.

Caution!!

Don't try to copy the .dmg file from one user to another on the same iMac.  Just download it again to the other user's desktop.

{....OK, Guille's addenda to Margie's comment. I don't know if dropping down dmg files all over the place creates havoc (although I wouldn't be surprised if it did).  This is what happened when Margie opened a dmg and then rebooted....}

I tried to open a copied .dmg, and ended up in some sort of  infinite loop when I tried to double click on the .dmg as the other user.  When I rebooted (about all I could do, or so I thought, since I couldn't kill the program) I ended up with the waiting for local disks forever screen.  I suppose it's because the .dmg is seen as a disk, and it wouldn't open.

Guillermo fixed this by unplugging our external hard drive, rebooting, then plugging it back in.

{...OK,  Guille's addenda continued...So I'm looking at this mac endlessly trying to do something with a local disk ...what exactly, god knows 'cause SJ feels it's none of my business & heaven forfend he make it obvious to set up a verbose boot mode (I'm sure there is one but I have no idea how to get that to happen)...so where was I, oh...I guess. If the disk system is the problem then maybe changing something like unplugging the external hd might do it. So to quote Bugs Bunny..."I dood it" and voila!...she continues to boot, goes to login dialog and I login. I plug the ext hd back and after a few tics she mounts. I then update the firefox by dropping the program into applications (after proper authentication of course)  and removing all signs of available firefox dmgs, do a restart and next thing you know the old imac is back to its hippie self (it is a flower power mac after all).

Finished by cleaning up the open dmg in Margie's account, and sure enough right version of firefox is up.

So to summarize: Mac's on OS10 go fubar periodically just like any other computer (I'm a great fan of "Crash Different")...I just wish I had some idea as to why?

... /end Guille's rant}

find syntax

from http://www.athabascau.ca/html/depts/compserv/webunit/HOWTO/find.htm#EX01:

find . -name "filename" -print

This command will search in the current directory and all sub directories for a file named filename.

tar with exclude, rm files starting with -

To tar up a directory without the subversion folders embedded throughout:

$ tar cvf file.tar directorytotar --exclude .svn

or

$ cat > exclude
.svn
[control-D]

$ tar cvjf file.tar directorytotar -X exclude

If you goofed and tried it this way first:
tar cvjf -X .svn file.tar directorytotar
tar --exclude .svn cvf file.tar directorytotar

You're then stuck with a file named --exclude or -X

You can't do

$ rm -X
rm: invalid option -- X
Try `rm --help' for more information.


Instead, do rm ./-X or rm -- -X

$ rm ./-X
rm: remove regular empty file `./-X'? y
removed `./-X'

from http://www.cyberciti.biz/faq/unix-linux-remove-strange-names-files/

Jared Richardson said...

At the NE-Jug meeting last night, Jared Richardson said to start a blog in which you write everything you've had to look up. So here it is: margie's tech blog.

margietech was taken.