SVN - unversioned contents in SVN-renamed directory

I noticed this using Tortoise SVN in Windows:

If a directory is renamed under SVN and you then update its parent directory, your current directory will be deleted, and a new version of the directory, with the new name, and all of its contents, will be downloaded back to your checked-out version of the parent directory.

This can be a real pain when there are a lot of big files that are deleted and then downloaded back again. Enough to rant about, and want a better way of implementing SVN Rename, but that's not the subject of this blog entry. The subject of this blog entry is:

If you have unversioned items in a directory that is in SVN, and you (or someone else) changes the name of that directory in SVN, then after the commit or update, SVN will leave a copy of the directory with the old name, containing the unversioned contents.

That is:

1. I have a directory: Dir-v, which contains versioned file File-v and unversioned file File-u.
2. Someone changes the directory name to newDir-v and commits the change.
3. I update Dir-v's parent directory.
4. I now have two directories: 
1. versioned newDir-v containing File-v 
• in Tortoise/Windows you'll see a green check mark on their icons
2. unversioned Dir-v containing unversioned File-u
•  in Tortoise/Windows, you'll see a blue question mark on their icons
5. I can delete the unversioned Dir-v after deciding what I want to do about File-u.

Why do I have unversioned files in my SVN directories? Well, there are some local files that I want to be categorized with, and able to find when I'm looking at other files that are in the SVN, but I don't think the other people in the project want these files in their copy of the structure. Is that bad?