Monday, January 8, 2007

Announcing HoboCopy 1.0.0.0

Over the last few months, since I first released HoboCopy, I've been tinkering with it based on feedback from a few users, as well as my own experience using it daily. Today I posted the results of that tinkering: HoboCopy 1.0.0.0. Yes, it's stable and complete enough that I feel comfortable with the 1.0 label. You can download it from SourceForge here.

 

The highlights of the new release include the following:

 

Fixed a nasty bug. OK, you know how I said that HoboCopy would work with VSSAPI-aware applications like SQL Server to get you a consistent view of application state? Well, it turns out that I lied. I had a bug with the part of the process called component selection, and as a result the old version of HoboCopy was always giving you crash-consistent copies…i.e. whatever the hell was on the disk. That has been fixed, and SQL Server (and other VSS-aware applications) will now correctly flush their state before HoboCopy copies the files.

 

Vista compatibility. As Vista is now my main day-to-day OS, this was a must. Sadly but understandably, using the VSS API requires administrator privileges, so I had to add a manifest to HoboCopy so it'll give you that little secure "running elevated" prompt when you run HoboCopy under UAC. This sort of sucks for a command-line tool, since it launches another cmd.exe, meaning any errors immediately disappear when the process exits. Maybe I'll figure out a fix for that in a future version. In the meantime, running HoboCopy from an elevated cmd.exe is a reasonable workaround.

 

Default non-recursive processing and filename filters. You can now run hobocopy C:\sourcedir C:\targetdir *.txt *.foo to copy only files with the .txt or .foo suffixes. Recursive directory processing is off by default, so you need to run hobocopy /r C:\sourcedir C:\targetdir *.txt *.foo  if you want to include child directories as well.

 

There are a few more minor things I'd like to add to HoboCopy - specifically, to make it asymptotically approach robocopy's capabilities - but at this point it works pretty darn well for 99% of what I want to use it for, so I'm not sure it'll ever make it to the top of the project heap.

 

One more note: if you have any trouble running HoboCopy, it's probably because you don't have the VC++ runtime DLLs on your system. Run vcredist_x86.exe and try again. I've made vcredist_x86.exe available for download in the same place as HoboCopy itself.

 

Anyway, download and enjoy. Let me know if you have any problems.

64 comments:

  1. God I love the name of this tool. ;)

    ReplyDelete
  2. Thanks - it was one of those odd moments of inspiration. :)

    ReplyDelete

  3. Doh!



    On several XP machines, when I try to do a copy, I get this:



    "

    HoboCopy (c) 2006 Wangdera Corporation. hobocopy@wangdera.com



    Starting a full copy from C:\Program Files\ExtractNow to C:\Documents and Settings\Administrator\Desktop\tmp

    There was a COM failure 0x8000ffff - .\HoboCopy.cpp (349)

    "



    :-( Any ideas?



    ReplyDelete
  4. " MS Software Shadow Copy Provider "

    needs to be set to be running before robocopy starts.



    Either set to AUTO to have it running always or call the service just before you run hobo with:



    net start <service name>

    ReplyDelete
  5. Ah, yes, of course - thanks!



    I assume, though, that you meant "hobocopy" and not "robocopy" in your first sentence.

    ReplyDelete
  6. Probably should add a check/warning for that in the code itself if you get a chance cause its not exactly a user friendly error message.



    I want to do some coding myself (have a few ideas to improve upon this great app. but work have me at my limits for the forseeable future.



    ReplyDelete
  7. That's a good idea.



    If you ever get around to making some changes I'd love to incorporate them. But I certainly understand being busy.

    ReplyDelete
  8. "The system cannot execute the specified program." on Server 2003 even after installing vcredist_x86.exe.



    Oh well.

    ReplyDelete
  9. I've got some free time coming up this month - I'll see if I can repro.



    In the meantime, if you want to run depends.exe against it to see if there are any missing DLLs, that would be enormously helpful.

    ReplyDelete
  10. When will I be able to copy mapped drives???

    ReplyDelete
  11. That would imply being able to create shadow copies on remote systems. I don't think that's going to happen.



    You can use robocopy now (although you won't get shadow copies) or you can run hobocopy on the remote system.

    ReplyDelete
  12. Still got the COM-error...

    I am running vista x64 sp1 and copied hobocopy in the system32-folder. When I try to run hobocopy I will get the following COM-error:

    There was a COM failure 0x8000ffff - .\HoboCopy.cpp (172)

    I AM an admin and I started the cmd via "run as admin" additionally.
    Also the Shadow Copy Provider service is up and running.

    Any ideas???

    ReplyDelete
  13. OK, the new x64-version solved the problem.
    Many thanks for that great tool!
    Now I can run the command like
    "hobocopy %temp% c:\deezer-music fla*.tmp"
    and then rename the copied file to *.flv.

    Nice job:
    Paul-O

    ReplyDelete
  14. Glad it worked! Especially since I have no idea what the problem would have been otherwise. :)

    ReplyDelete
  15. Hi, well done for the great tool!

    The source compiles on VS8, but the exe does not run.. I get:
    "The application has failed to start because stlport.5.1.dll was not found. Re-installing the application may fix this problem."

    Visual Studio complains with:
    "Debugger:: An unhandled non-continuable STATUS_DLL_NOT_FOUND exception was thrown during process load
    The program '[94712] HoboCopy.exe: Native' has exited with code -1073741515 (0xc0000135)."

    And, also depends.exe complains about two dll's:
    STLPORT.5.1.DLL (as expected) and also DWMAPI.DLL

    Any clues?

    ReplyDelete
  16. PS I installed vcredist, and still got these errors at runtime.

    ReplyDelete
  17. So you're compiling from source? Have you tried the binary distribution? Hobocopy is sort of in a holding pattern for me (two kids, no time), so I'm hesitant to try troubleshooting your compilation setup.

    ReplyDelete
  18. No problem craig, the binary works fine.. just compiling from source to learn a thing or two about VSS... Probably something wrong with my PC setup / VS installation... will try on another PC..

    Thanks once again for this neat tool.

    ReplyDelete
  19. @Julian: glad to hear the binary works. If you get totally stumped, feel free to fire a question my way, although I can't guarantee I'll be helpful, or fast...I know just enough C++ to be dangerous. Which also correctly implies that my C++ code might not be the best learning tool. :)

    ReplyDelete
  20. Hi Craig, thanks.. I'm tinkering around with a hobby project that might need to use VSS, so I just might take you up on that offer if I'm in a total fix on some part or other of the code... :)

    On a different note, it seems your choice to use C++ was in the long run, the right one... I found this on one of the comments on this site:

    http://www.codeproject.com/KB/dotnet/makeshadowcopy.aspx?fid=966755&df=90&mpp=25&noise=3&sort=Position&view=Quick&select=2496895

    Not sure how reliable this info is, but it seems to make sense in view of the lack of documentation:


    I went to the work of contacting the Microsoft VSS team. I was able to create the interop for vista and 2008 however in vista and 08 the vss coordinator class is lacking the ability to create and do snapshots along with a few other things. I was informed that the vss coordinator class is unsupported and was only made for internal use thats why its not documented anywhere on msdn. The only way to do it for 08 and vista is to write something in mc++ and make calls to it. The correct class to use is the IVSS coordinator.


    What do you think?

    Julian

    ReplyDelete
  21. I looked at VSSCoordinator and it seemed to be pretty sparse - I don't think you can do much seriously with it. C++ would appear to be the only way to go. Whether you use straight C++ or Managed C++ is a question of taste and ability - you could write a straight C++ DLL and use PInvoke to call into it from a .NET app.

    ReplyDelete
  22. Craig, thanks for your reply. Managed C++ sounds like the cleanest option, since the rest of the stuff will probably be written in C#.
    Not too sure here (I never used managed C++), but this is obviously assuming that the VSS SDK can still be used through managed C++ right?

    ReplyDelete
  23. MC++ should be able to do anything unmanaged C++ can do. Although I admit it has been years since I've worked with MC++ and didn't do much even then. Still, I'd be surprised if it couldn't handle this.

    ReplyDelete
  24. It wont start, the command prompt window appears for a split second, then closes before you can read what it says.

    ReplyDelete
  25. Just wanted to mention my attempt at a .NET interface to the VSS SDK, using managed C++ to expose the full VSS API. Still in early alpha, but early tests seem promising. It is freely available at http://www.alphaleonis.com/

    I hope this will help finally solve these problems we all seem to be having with accessing VSS from .NET.

    ReplyDelete
  26. This looks like an awesome tool, but unfortunately I run into an error when I try to use it. I have Windows XP SP2, and I have downloaded the XP version and even compiled it with the same results. No matter what file I try to copy I get the following error:

    hobocopy /full c:\temp\somefile.txt d:\temp\
    HoboCopy (c) 2006 Wangdera Corporation. hobocopy@wangdera.com

    Starting a full copy from c:\temp\somefile.txt to d:\temp
    DoSnapshotSet failed.


    FWIW, the "MS Software Shadow Copy Provider" and "Volume Shadow Copy" services are both running and I have the vcredist_x86.exe is installed.

    Any suggestions?

    ReplyDelete
  27. I used this before a couple days ago to do copy some files, now when I try to do it again I get the following error:

    Starting a full copy from C:\DOCUME~1\Vindra\LOCALS~1\Temp\ to C:\bckup\
    There was a COM failure 0x80042316 - .\HoboCopy.cpp (330)

    ReplyDelete
    Replies
    1. I got the exact same error on XP. Restarting windows' shadow copy service (through running services.msc first) solved the problem. Good luck!

      Delete
  28. This could be caused by hobocopy crashing (or being aborted) halfway through and leaving a dangling shadow copy. Much as I hate to suggest it, a reboot might fix this.

    ReplyDelete
  29. for the 330 error, check if there are any .tmp files in the working directory of hobocopy and delete them. These were created because I ran incremental copy before I realized I had to run full copy on first run.

    ReplyDelete
  30. I seem to be having a problem when the directory has spaces in the filepath. Such as C:\Documents and Settings\. It will create a directory named "and" in my destination path while saying to copied from C:\Documents. Program works great on paths without spaces. Such as C:\temp\. Using Win XP Pro SP2. Thanks

    ReplyDelete
  31. i m facing this type of problem please give me the right solution:
    "snapshotSet failed" when i run the hobocopy.

    ReplyDelete
  32. dear sir,
    I have wrote before that i m facing this type of problem " DosnapshotSet failed" you didn`t gave me any solution ! please give me the right solution that`s why i will be able to use that software......... Its a request sir...

    ReplyDelete
  33. I have Windows XP SP2, and I have downloaded the XP version and even compiled it with the same results. No matter what file I try to copy I get the following error:

    hobocopy /full c:\work\somefile.txt d:\work\

    HoboCopy (c) 2006 Wangdera Corporation. hobocopy@wangdera.com

    Starting a full copy from c:\work\somefile.txt to d:\work

    DoSnapshotSet failed.

    Both "MS Software Shadow Copy Provider" and "Volume Shadow Copy" services are both running and I have the vcredist_x86.exe is installed.

    ReplyDelete
  34. Dosnapshotset failed.

    Check the partition type it should be NTFS. For converting FAT32 to NTFS
    run c:\convert c: /fs:ntfs

    ReplyDelete
  35. Hi all,

    I have this error "There was a COM failure 0x80042301 - .\HoboCopy.cpp (196)"

    I am running under Win XP Home SP3.

    Thanks in advance

    ReplyDelete
  36. Hmm. That's E_BAD_STATE, which can mean any number of things. Can you grab a copy of vshadow.exe from somewhere and run

    vshadow /tracing

    ? More info here http://thorny-issues.blogspot.com/2008/11/volume-shadow-copy-service-vssebadstate.html

    ReplyDelete
  37. any solution to the dosnapshotset failed problem?

    ReplyDelete
  38. Hobocopy is no longer in active development until another developer steps up to take over the project. So I don't have a solution for you on the dosnapshotset failure.

    However, someone else suggested that this may be due to running hobocopy against a FAT partition, instead of an NTFS one. Are you trying to snapshot a FAT partition?

    ReplyDelete
  39. Hi, I also have the problem as lt8nk has, the COM failure 0x80042310. Any ideas?

    ReplyDelete
  40. Hi all!

    I have the same error as I8nk.

    I have this error "There was a COM failure 0x80042301 - .\HoboCopy.cpp (196)"

    I am running under Win XP SP3.

    Thanks in advance

    ReplyDelete
  41. Running Hobocopy on Svr 2003 I get "The system cannot execute the specified program."

    Dependancy Walker output:
    Starting profile on 12/15/2009 at 12:21:29 PM

    Operating System: Microsoft Windows 2003 Domain Controller, Small Business, Small Business Restricted (32-bit), version 5.02.3790 Service Pack 2
    Program Executable: d:\HOBOCOPY.EXE
    Program Arguments: /?
    Starting Directory: D:\
    Search Path: C:\Program Files\Support Tools\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\Microsoft SQL Server\80\Tools\Binn\;C:\WINDOWS\ADMT;C:\Program Files\Microsoft SQL Server\90\Tools\binn\

    Options Selected:
    Simulate ShellExecute by inserting any App Paths directories into the PATH environment variable.
    Log DllMain calls for process attach and process detach messages.
    Hook the process to gather more detailed dependency information.
    Log LoadLibrary function calls.
    Log GetProcAddress function calls.
    Log debug output messages.
    Automatically open and profile child processes.
    --------------------------------------------------------------------------------

    Failure starting the process. This application has failed to start because the application configuration is incorrect. Reinstalling the application may fix this problem (14001).

    ReplyDelete
  42. I run hobocopy on server 2003 nightly, so it can be made to work.

    You need the VC runtime redistributable and the correct version of hobocopy, both of which are available on the SourceForge download page.

    ReplyDelete
  43. I'm trying to implement VSS into a backup project I'm working on. Aside from the general confusion on how to use VSS inside of the C++ project, I'm running into an issue with how I backup the files.

    I search for the files to be backed up, then pass the path to a compression function. The function simply calls zip.exe with the arguments. From simple tests in command prompt, I have seen that it can't directly access the \\?\GLOBALROOT\... path. Is there a way to expose this path to external programs? I'm not sure how I can use VSS to copy locked files (registry, databases, etc). The machines are running Windows XP and Server 2003.

    I don't want to simply copy the files/directories as some of the databases are quite large and it would add a lot of overhead. How can I access the shadow copies in my code or an external program?

    zip.exe is Info-ZIP.org's program.

    ReplyDelete
  44. You should be able to mount a drive letter (e.g. Q:) at \\?\Globalroot\... I don't know what the API for that is, but it should be easy to find. The only caveat is that with non-server versions of VSS, the shadow copy disappears from the namespace once the program terminates, so if you spawn any external processes, the process that initiates the shadow copy will need to stick around.

    I've been thinking about making a tool that would do more or less exactly this for some time now.

    ReplyDelete
  45. Starting a full copy from c:\WINDOWS\explorer.exe to C:\explorer.exe
    Calling CreateVssBackupComponents
    Calling InitializeForBackup
    Calling GatherWriterMetadata
    Waiting for writer metadata
    Calling QueryStatus for GatherWriterMetadata
    Calling GetWriterMetadataCount
    There was a COM failure 0x80042301 - .\HoboCopy.cpp (196)

    Running on Windows XP Home Edition SP3 with Visual C++ 2005 and 2008 SP1 installed...

    Awesome tool otherwise, will utilize somewhere else - hopefully.

    ReplyDelete
  46. Have you tried to run vshadow /tracing? Are there any entries in the eventlog?

    There are also some suggestions here: http://www.pcreview.co.uk/forums/thread-1478541.php

    ReplyDelete
  47. I too ran into the "COM failure 0x80042301 - .\HoboCopy.cpp (196)" problem. Hobocopy worked fine after I deleted the following registry subkey:

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\ContentIndex\Catalogs\Visio

    I didn't have Visio installed on my system, and that was causing a problem. You should look under at the Catalogs key and remove any subkey whose program is not installed on your computer -- visit the Microsoft Support page for details:

    http://support.microsoft.com/default.aspx?scid=kb;en-us;907574&sd=rss&spid=3223

    Craig: much much thanks for sharing this nifty little program :)

    ReplyDelete
  48. @Voon: Awesome! Thanks a ton for posting this - I hope it helps a few people out.

    ReplyDelete
  49. First of, thanks for an awesome tool, it's immensely useful!

    I also ran into the "DoSnapshotSet failed" and for me it was caused by open virtual drive created by TrueCrypt. It seems that HoboCopy or one of the services it uses internally tries to enumerate all drives (I see my floppy light come on), and chokes on this particular virtual drive. Once I closed it down the error went away...

    ReplyDelete
  50. I also had the "DoSnapshotSet failed" error. But the funny thing is that I only get it some times (and I run it two different computers, both with XP Pro.)

    It seems that when I do get the error, closing the Truecrypt drive(s) fixes it. Note, it only hiccups sometimes on the drives, not all the time.

    ReplyDelete
  51. I got the "COM failure" error, and I managed to fix it by installing the VSS SDK 7.2. You can get it here:

    http://www.microsoft.com/downloads/details.aspx?FamilyID=0B4F56E4-0CCC-4626-826A-ED2C4C95C871&displaylang=en

    ReplyDelete
  52. Hello, I'm getting the following error and all the fixes above, won't work:
    There was a COM failure 0x80070005 - .\HoboCopy.cpp (169)

    Any ideas?

    Thanks

    ReplyDelete
  53. What output do you get when you run it with /verbosity=4?

    ReplyDelete
  54. Hello Craig,
    the following message appears:

    Calling CoInitialize
    Starting a full copy from C:\windows\system32\ to D:\temp\
    Calling CreateVssBackupComponents
    There was a COM failure 0x80070005 - .\HoboCopy.cpp (169)

    Thanks,
    Tim

    ReplyDelete
  55. Is the volume shadow service running?

    ReplyDelete
  56. Yes, I've started it with
    "net start vss"

    ReplyDelete
  57. Have you verified that it stays started via the services control panel?

    ReplyDelete
  58. Yes it is running. But one thing that probably causes this error is that I'm a normal user, not an admin (although I work with XP)

    I just read that under Vista: "VSS API requires administrator privileges" - so this is probably the same for XP?

    ReplyDelete
  59. I have tested HoboCopy on XP (although it has been ages since I did anything with it), but I don't recall whether I was using a user in the administrators group. Obviously, the whole elevation thing doesn't exist there.

    Is there any way that you can try it as an admin user and see if that clears up your problem? I'm not sure I have an XP VM handy right now.

    ReplyDelete
  60. OK, as part of my effort to actually put some (not much, but some) time into Hobocopy, I'm moving it to GitHub, and I've started a mailing list at hobocopy@googlegroups.com. I'll direct future conversations there, since the comment thread on a blog post is a pretty bad medium for a support forum.

    ReplyDelete
  61. Hi,

    I am getting the below mentioned error on copying a file.Please help me.

    "There was a COM failure 0x80042301 - .\HoboCopy.cpp (196)"

    ReplyDelete
  62. Hi! I've recently made some changes to the project structure. Please make support requests either via the project group (http://groups.google.com/group/hobocopy), or by reporting an issue on the GitHub project site (https://github.com/candera/hobocopy/issues).

    Thanks!

    ReplyDelete
  63. the solve to:
    Calling CreateVssBackupComponents
    There was a COM failure 0x80070005 - .\HoboCopy.cpp (169)

    is to run hobocopy as administrator. with normal privilges windows doesnt allow the user to start Volume Shadow

    ReplyDelete