[Yanel-dev] Fwd: Cluster documentation

[Cedric Staub]

On Wed, Feb 02, 2011 at 06:21:33PM +0100, Michael Wechner wrote:
> Hi Balz
> 
> On 2/2/11 5:55 PM, Balz Schreier wrote:
> > Hi Michael,
> > interesting point is mentioned in the restrictions:
> >
> >     * Only one cluster node writes to the repository. This means that
> >       the authoring environment is only accessible on one dedicated
> >       cluster node, which is called the /master/ node.
> >
> >
> > For the project I'm working for this is not a solution as each node 
> > must have write access to the repository as well as to the Lucene index.
> 
> agreed and I don't think you need to worry
> 
> > This is what I wanted to highlight in today's workshop that 
> > Yanel/Yarep should come up with a concept of a single "write" instance 
> > which gets used by all members in the cluster so that writing is also 
> > possible for other nodes... to be continued....
> 
> wyona.com is running within a cluster and so far we didn't have any 
> problems.
> Also please see Cedric's tests, whereas to be fair these tests were done 
> only with concurrent read access.
> But we will also do concurrent write/read.

I don't think doing concurrent writing with NFS will work though.
We have to make sure only one process can write to a file at once.

Some of my thoughts about possible solutions:

* Build a locking mechanism into Yarep. This would solve the problem,
  but I think it will be very difficult to build a good, solid locking
  mechanism.

* Simply use a backend that can do concurrent writing, e.g. a good
  distributed filesystem or a database. It's easier, but not always
  practical, e.g. what if someone has already deployed a given 
  filesystem somewhere and we can't change it etc.

Just my thoughts,
Cedric