so I've acquired a server - now what? Help!

[Mac]

I've moved away from RAID/drive redundancy setups for a few reasons. Firstly, they can be a a bit costly, a bit fiddly to setup....and let's say for example a specific *box itself* fails. Good luck taking all of those RAID'd drives and putting them in another unit - even of the same manufacturer - and them working. 

 

It's a lot of effort, and not as safe as you think.

 

Most platforms now are moving away from RAID as a protection mechanism and using replication. It's simpler, safer, and mostly cheaper too. Sometimes RAID is used where performance is required of course, or if you want additional protection on top of replicated data.

 

So, personally, I go for JBOD or Just-a-Bunch-of-Disks. OMG WHAT IF ONE FAILS you say. Well, I replace the drive, and replicate it back from one of the two other stores.

 

It's simple. What's more I can physically take that drive plug it in to any other SATA/eSATA store and it'll just work. No messing.

 

All major platforms are going that way now. MSFT Exchange did it a few iterations ago - shared-nothing clusters (or Wolfpack) were an utter pain in the arse. Now they just use file/block level replication to any old storage, and hey, it just works, and it's far easier to manage and recover from than traditional clustering/RAID type setups.

 

If anything it introduces another problem - often it's so damm resilient that sometimes clients don't realise something has broken. So they carry on regardless...and then eventually something does break, and it's then they found out they lost their two layers of resiliency. Monitoring is absolutely key in such setups.

 

I'm boring myself.

Edited September 29, 2015 by Mac

[Mac]

So...to relate this to my 'home' setup rather than my work stuff. I pay attention to budget for my home stuff, but bizarrely I also give it a higher factor of backup because hell I can't recreate the pictures and videos of my lass when she's growing up, but I can rebuild work boxes.

 

I have a primary stack of drives - all single instance drives. No fancy RAID. All plugged in via Thunderbolt, although they can also be accesses via a small Mac Mini server over WiFi if I like, same for iTunes etc. Without looking, I think they're 4 x 5Tb drives (I've a lot of stuff, and to be fair there's a lot of old legacy work stuff I've never gotten around to deleting). 

 

The stack of drives is live-duplicated to TWO other stores. So for every 1Tb drive I need 2Tb backups. The 3rd of those 3Tb though is 1 week time-lagged behind the primaries. 

 

On top of that, I have TimeMachine backups for the primary store (hourly, changes), and I also have an online store via Backblaze for all my critical stuff. The Backblaze is 'point in time' storage (PIT), so I have to physically put stuff there for it to be uploaded. I do this every few weeks with photos, email, stuff that changes.

 

Lot of protection there, but it's far simpler than I've probably made it sound. I could pick up any one of those drives and plug it in to any other machine, and as long as I know the encryption config it'll just work.

[Waylander]

Brilliant thanks.

So ignore the shr and read jbod.

Is the rest of my architecture okay?

[Mac]

Well....I'm not quite sure I understand what you're trying to do But, from a principals point of view, you really want 3 instances of your data, and a 4th off site somewhere.

 

In Office365 for example, there's six instances. There was loads of work done that seemed to point this being the optimal number, have 6, 2 of them time lagged, and you'll never lose data again.

 

Or something. I wouldn't put anything past you

[Waylander]

I'll have at least 3 instances: Local on pc, local on server, local on external hdd connected to server.

Then back blaze from server (for photos/data) and OneDrive from pc (for data).

I actually have another 320g external drive so I'll chuck the pix and home movies on there and stash that at parents.

[NewNiceMrMe]

Is the data really worth all the effort?

 

Really?

 

I have my data on my MBP.  I have it on a 4K iMac 27.  I have it in Dropbox.  I then have photos automatically backed up to Amazon and Google.

 

It all happens seamlessly and was all set up in minutes.

 

Office365 takes care of mail.  End of.  No worry, no stress, no complex configurations.

[Waylander]

What's the effort?

I want a media server anyway so it'll have more than enough space to backup data and pix.

The rest of it is just a few clicks and a couple billion questions on here and then it's automated.

The effort really will be setting plex and the shares.

And therein is the rub - this thread is basically a reflection of my excitement at actually having a new toy and new geeky tech

Edited September 29, 2015 by Waylander

[Mac]

Is the data really worth all the effort?

 

 

 

Yes. Yes it is. I don't really care about reproducible stuff - but everything else? I avoid physical 'things'. I don't print out photos, I don't have 'stuff'. Friends who come to my house have been known to make the comment that my house looks like nobody lives there. I get that.

 

I don't particularly care about reproducible data. My work stuff for example has one nearly line backup. Worse case  is both of those die, I have to drive to Reading.

 

My personal stuff - my photos, videos, personal diaries and vlogs - no, they're irreplaceable. I'll protect them to the point that the only way I could lose them would be through something utterly catastrophic.

[CarMad]

Just to comment I'm the same as Mac in that I don't use RAID for the same reasons, I've just got a 4 bay USB3 external enclosure for the time being and another at another location. 

 

I replicate the data using another part of crashplan that is free that sends the data to another location. 

 

Critical bits also get sent up to crashplan as well as another separate drive for photos and videos only that goes to my parents. 

 

The thought of losing all of my photos or videos is not something I could ever contemplate. 

[NewNiceMrMe]

Yes. Yes it is. I don't really care about reproducible stuff - but everything else? I avoid physical 'things'. I don't print out photos, I don't have 'stuff'. Friends who come to my house have been known to make the comment that my house looks like nobody lives there. I get that.

 

I don't particularly care about reproducible data. My work stuff for example has one nearly line backup. Worse case  is both of those die, I have to drive to Reading.

 

My personal stuff - my photos, videos, personal diaries and vlogs - no, they're irreplaceable. I'll protect them to the point that the only way I could lose them would be through something utterly catastrophic.

 

But that applies to me too and the solution was created in minutes.  Amazon Photos, Google Photo's and a back up on two other physical devices (oh and the Apple Cloud account syncing to the phone and iPad of course).

 

I'd need Amazon to go bust, Google to go bust (or servers to go bang), and my Mac's to go bang or up in flames (or drowned before someone else says it).

 

My point being, is such vast debate and technical complexity really necessary?  I don't think so. 

Edited September 30, 2015 by NewNiceMrMe

[Mac]

Ever tried uploading 20Tb of historical stuff to Amazon?

[Waylander]

There are some incredibly proficient people on this site; this is new toy syndrome for me - it is as much about learning the ins and outs about such things from people as anything else....as I've said before.....and the setting up of a home streaming server (which I accept you have via some apple thingumijy, probably)

 

So yes - a few pages, lots of interesting (for me!) chat and at the end the solution is the simple one CarMad proposed on the first reply!

[Tipex]

All this home streaming stuff, how many people actually stream media from their own storage space?

 

Does anyone actually buy films digitally? or music?

 

Everything we watch that isn't live TV is streamed from Netflix or Apple Music, (other providers are available), anything I want to actually own is bought on physical media, i.e. Blue Ray, DVD or CD, I very very rarely purchase digitally, due to the fact I know Apple wont be my provider of choice for ever, they will be overtaken at some point, and when I no longer use their products, I wont use Apple Music, so will have effectively 'lost' anything I've paid for, whereas a CD is universal and will last as long as I still have a CD player, and I can also rip it and use it in any digital library anyway.

 

I don't really understand the need for large volumes of storage space for anything other than personal photos/video/work.

 

Having a home server seems a bit old hat these days really when everything (media) is streamed from the cloud?

Edited September 30, 2015 by Tipex

[Waylander]

Depends how you get it I suppose especially with many US tv shows

[Tipex]

Well personally I don't bother streaming anything illegally, as anything worth watching will always make itself available sooner or later.

[NewNiceMrMe]

Ever tried uploading 20Tb of historical stuff to Amazon?

 

That's a fair point, but everyone has to start somewhere!

[NewNiceMrMe]

We use digital streaming to a massive degree.  MrsMe is possibly the only person in the house to watch live TV (other than when I watch football).

 

However, if I buy a movie, I never hold a copy locally.  I don't see the point.  I've watched it.  Over and done with.

 

That said, different people have different preferences and lifestyles.  If it works for them, so be it.

 

Same with cars.   Some people still buy Audis.  Go figure.

Edited October 1, 2015 by NewNiceMrMe

[CarMad]

Some of the streaming services are poor in terms of quality and on occasions sound. 

 

I've left the NowTV as it looked terrible on my TV. It was only 720p and a very low bitrate and it was bad enough for me to not want to pay for it. 

 

Amazon doesn't seem to suffer anything like as much and I've not tried Netflix to comment. Much of our streaming is from the kids directly from the TV another thing NowTV couldn't do. They don't support an application on Samsung TVs either which was another nail in the coffin for them. 

[Tipex]

Netflix will stream 4K if you have the equipment and a fast enough internet connection.

I've certainly never noticed any image or sound quality issues watching in HD, looks great.

[Waylander]

well its arrived as have my 2 WD Red 3TB drives

 

I have clearly misunderstood what JBOD did as I thought it was basic drive access but I found it merged everything into 1 volume so I have set them up as 2 basic volumes (one will be for multimedia and 1 for backups etc)

 

updating and installing Xpenology was very straightforward so currently adding packages.

Will bung it in the loft soon.

 

need to read up setting up couch potato and sickrage.

then have to actually get me a new pc!

[Waylander]

I got so sucked into fiddling with this, I completely forgot to buy my PC components!!

[Riz]

[Waylander]

I tell you what - initially setting up media shares from the PC were a piece of p*ss

Now that I want to completely hand off to the server, removing the shares (from the TVs) has been bloodt fiddly!

[NewNiceMrMe]

I tell you what - initially setting up media shares from the PC were a piece of p*ss

Now that I want to completely hand off to the server, removing the shares (from the TVs) has been bloodt fiddly!

 

I just cannot fathom all the effort you're going to!  Are you holding copies of the national archives for the government and being paid for it?  I'm guessing you must be because of the lengths you're going to!

[Waylander]

I find it fun to do and learn about (shrugs)