The Great Synchronization Thread

[csaba]

Hello Guys and Girls,

It has been obvious to us, that the most needed and asked-for feature in ForkLift is the possibility to synchronize folders.
We are now happy to announce that we have started development of our synch module.
As we want to make this the best damn synchronization tool that ever was, we are asking You, our Esteemed User, for input on what you would expect from such a tool. To this end, we are now launching this new thread in our user forum, and encourage you to make yourself heard.

What we are looking for most, is descriptions of the tasks where you need to use a synchronization tool; and your ideas about how you would expect to manage the workflow of these tasks. We would like this to be a very open and active discussion, one that we hope will result in a tool worthy of the general excellence of ForkLift.

We will hear out any ideas, however weird they may seem You are welcome to post screenshots, diagrams, filelists, links, anything useful to this discussion.

Thanks for contributing

BinaryNights

[Adrian]

A fully-fledged synch system sounds great, but I'd like to suggest a few really simple things that would be a big help for uploading to a remote sites.

Being able to specify a "local path" for a connection is great; additional options for the connection would be:
- turn on sync browsing automatically
- don't confirm overwrites on the remote server
- force the remote server into the right-hand pane, or the left - the point is it should be the same every time. I have inadvertently overwritten local files with remote ones because I had focus on the wrong pane when I opened the connection. Remember, both sides have the same files so look almost identical!

I guess what I'm getting at here is that while full automatic synching will be enormously useful at times, streamlining the process of uploading just a couple of files would be great too.

Thanks for Forklift. Loving it.

Adrian

[yevgeny]

* Syncing between two remote servers: Amazon S3 Euro & US buckets, different SSHs.
* 'One master, multiple slaves'.
* A scheduled synchronization (in background).
* applescriptability.

[buddha379]

To start, a quick aside: thank god for Forklift. I came from a Windows desktop. As such I can wisk files around in Windows Explorer like it's an extension of myself. I love my Mac, but Finder is an atrocious piece of crap. Having dual panes is the only way to go, if you have network shares, or multiple network shares like I do. Anyway, on to sync.

First off, not sure there is really a lot of need to reinvent the wheel here. Rsync is a fantastic program and can do two way synchronization (EDIT: no it doesn't, only unidirectional, see edit notes below, though you can hack a two way sync with -u option, which says don't update newer files, which allows you to do sync up and then sync down, in essence performing a bidirectional sync based on date). Most importantly it can do it over SSH, so communication is secure. I say just do a pretty front end to that and hide the complexity from the user. Only downside is that it can be a bit sluggish with lots of files, building its file list.

I really like the scheduled sync idea from Yevgeny.

As for Applescript, I'd be happy just to access it from bash with a nice simple command utility, that I could call in a shell script.

Sync should also have push, pull or bidirectional sync. Push to push up to a server. Pull to yank down. And bidirectional to make sure that only the latest in both directories are copied to both the local and remote server. Bidirectional is the most important. I have a CentOS server at my office that I sync to manually now. Would be a lot easier if it was push button.

Sync should also run as a separate process, perhaps as a daemon, that can be kept running in the background, even if the user closes Forklift. Of course the user should have the option to shut it down or daemonize it when he is closing Forklift, or he should be able to set it in preferences, i.e. "always run in the background" or "only run when Forklift is open" or "prompt me to choose," something like that.

Edit: I tested Unison which does bidirectional sync, but it's codebase is stale ( no longer updated ), not to mention it's written in some funky language called Camel, which may be a cool language, but I wouldn't know because I've never seen anyone use it! So you may need to reinvent the wheel here, unfortunately, unless you expand rsync or use the -u option.

Edit 2: I just went through the unison install on my Mac and on my Linux box. What a huge pain in the butt! And because it's written in the never-heard-of-it-before language "camel", I think using it as a potential backend is completely worthless.

Edit 3: There is an opensource program called Cyberduck, but from what I read in the forums Sync crashes it regularly and is horribly broken.

[cintra]

First off, not sure there is really a lot of need to reinvent the wheel here. Rsync is a fantastic program and can do two way synchronization. Mosty importantly it can do it over SSH, so communication is secure. I say just do a pretty front end to that and hide the complexity from the user. Only downside is that it can be a bit sluggish with lots of files, building its file list.

ref. the above, I was going to say that in addition to my usual SuperDuper and Time Capsule backups, I use arRsync to sync half a dozen special areas.

You should at least be able to match the simplicity and exceed the features of arRsync, which is a very simple front end to Rsync..

mvh

[edified]

I agree about using rsync. You'll benefit from its maturity and ubiquity. Just make sure to have a better interface than arRsync. It's good software but the GUI could use some serious rethinking.

[honzab]

Hi, first of all thanks for ForkLift. It's absolutely great.

Speaking of synchronization, I am used to use a commander called "Total Commander" on Windows, which I find so far unbeatable windows-tool. It supports synchronization of files using the synch view (see attachement of this post) which I find pretty cool.

So if you want to grab more inspiration for ForkLift's synch tool, try to look at http://ghisler.com where you can download a shareware version of TotalCommander.

Hope it helps a bit, because I would really love to see this feature in next version of ForkLift

EDIT1: It seems that the attachement got somehow thumbnailed, so here's link for it.

[funtuva]

I Agree 100% with other posts that Total Commander is so far unbeatable wiws-tool. It supports synchronization of files using the synch view (see attachement of this post) which I find pretty cool.

The sync windows is elegant, simple and VERY fast. It offers all possibilities, copy left to rigth, viceversa, show singles, duplicates, etc... I have tried other Windows tools (Viceversa, etc...) and Mac (Syntevo´s Smart Synchronization is the best, although I do not need the line by line comparison nor the 3 way comparison...) , ChronoSync is bloated and messy, File Synchornization is simplistic, Synchronize Plus...more of the simplistic, Synk Standard
is amongs the best but it leaves you with hundreds of conflicts to resove manually, although the approach is on the rigth direction.

My point is why search any more: Total Commander has been around for more than a decade in the form of Windows Commander, etc.. it is a fantastic 2 pane file manager, and the sync modulo is hard to beat, so I would LOVE to see it come just the way it is into Forklift. I cannot think of improvements, really. It just works. So Please...........give us something like that!!

[rthrash]

* Syncing between two remote servers: Amazon S3 Euro & US buckets, different SSHs.
* 'One master, multiple slaves'.

Definitely these ... including being able to sync/mirror mixed protocols: SFTP <--> FTP, S3 <--> SFTP, etc. (and obviously local, too)

[George the Flea]

My most common sync activity is Mac -> FTP server while working on a website. Usually I'll do this while working with PHP (could use MAMP, but it's difficult to get MAMP set up just like the target server, and sometimes the work necessary to migrate an installed PHP program isn't worth the trouble).

Automatic syncing (watching a folder and pushing changes to the server without needing input) is a nice idea, but realistically I want control. Sometimes I'll save a file midway through, and uploading it at that point would just break things.

I completely agree with Adrian who asked for the panes to be locked in while working with mirrored directories. Accidentally overwriting a local filed while mirrored with a remote directory can be catastrophic.

What I'd ideally like to do is the first time I sync have the program ask me whether I'm uploading, downloading, or mirroring (or whatever terminology you use here; push/pull works just as well). After I've designated which direction I'm working in for the session, I'd like the program to maintain a visual indication of which files are going to be synced in the main window (little arrows, icons, color-coded backgrounds, whatever). There should also be some way to temporarily or pseudo-permanently exclude a file from syncing, and then I'd hit the sync button and have the changes run immediately. (Obviously some way to change the settings would be necessary, too; say if I wanted to go from mirroring to pushing).

What I would not want to see is something like Yummy FTP's workflow, where you click sync, adjust settings in a sheet, wait while it scans the directories, adjust/confirm which files are sent in another sheet, then finally see a sheet with a completion report (some steps of this can be disabled in the original settings sheet, but still). This takes far too long and really breaks up my workflow.

I've tried using Changes for keeping things synced up, but its connection is too slow (I've never had much luck with MacFUSE). That kind of an interface, where you can see at a glance what's being updated, is what I'd love to see, though.

Probably be a major interface headache to implement, but such is my perfect world.