Installing BrightStor
ARCserve 9.0
(backup software)
|
12/20/2005
BrightStor Arcserve 9.0 requires Btrieve 7.94 (Pervasive.SQL 2000i), of which a 2-user license version is installed with Netware 6.5. The BrightStor installation will also upgrade you to version 7.94 (with only the 2-user licenses) if you have an older version of Btrieve on your version of Netware. However, if you do have an older version and BrightStor upgrades it, you will lose any extra licenses that you might have had for the older version. We had purchased Pervasive.SQL 8 (Btrieve 8.00) with 10 user licenses, so that we would not be limited to just the two users, so I needed to upgrade it before I installed BrightStor. Btrieve 7 and above loads according to settings in the bti.cfg file and ignores any command line parameters. You may check btrieve paramaters by loading BTRMON.NLM (be sure to load btrieve and bspxcom first). Here is a little bit about Btrieve 7 settings. Here are the settings for use with Arcserve. Netware 6.5 SP4 comes with Pervasive SQL 2000i, 2-user license, which includes Btrieve v7.90.000. nwmkde.nlm showed version 7.94.251.000. The version number for btrieve can be checked by typing modules btrieve at the server console. Here is a good document about checking the Btrieve/psql version number. Supposedly, you can check your btrieve licences by typing NWUCINIT -g11. However, my copy of nwucinit did not seem to have the -g[code] option. Perhaps this was only v7 of psql. Supposedly there is a CLI License Administrator Utility that can do the same thing by running clilcadm. I did not seem to have that program. According to CA:
Apparently, btrieve uses SPX connections. Btrieve is started when the server starts, by running BSTART. If you try to run this manually, you can verify that btrieve is loaded (the modules will tell you that they won't load multiple). There is a file, PVSW.LOG that logs errors. We bought Pervasive SQL v8 with 10 licenses. The Pervasive SQL v8 User Guide can be found here. Apparently, version 8.5 and, subsequently, 8.6, are simply security updates to version 8.0. First, I backed up the \\sys\pvsw directory, which contains the existing Pervasive SQL licenses. However, if this is really v2000i, they're not upgradeable anyhow. You must do the server installation from a Windows client (logged into windows with Administrator privileges). Before you start the setup, you must be an administrator of the server object (I assume they mean Supervisor Object Rights). You also need Supervisor File System Rights to the SYS: volume, but supervisor object rights to the server would give you those anyhow. You must have the NLM's already loaded for whatever protocols you wish the clients to be able to use in the future (SPXS.NLM for SPX), and lastly, you must have a drive mapped to the SYS: volume on the server. I could check off all of those requirements, so I inserted the installation into my client's drive. I entered the License key and then selected Custom Install, so that I could select to install client installation programs for DOS as well as Windows. I left the default directory for the system files, \\SYS\SYSTEM, and the default directory for the rest of psql, \\SYS\PVSW. I checked all of the install options, and clicked Next. Oooh, this is fun:
I clicked OK and got the following message:
I looked at \\SYS\PVSW\INSTALL.LOG, and it looks like the installation failed when it was trying to install some stuff local to Windows. Hhhmmm... I realized that I had NOT typed BSTOP on the server
before I had started the installation, so I did it not. It turned out
that the server had some database files open! Then, I did what it said
and rebooted, then uninstalled Pervasive SQL from the Windows client.
At this point, I noticed that this removed psql from the server as
well, and neglected to roll it back to the previous version - so
now, there was NO psql/btrieve installation on the server at all! Nonetheless,
I rebooted the client and then started the installation again. This
time, it completed successfully! At the end of the install, it gave
me this message:
After I rebooted the server, I noticed that btrieve was already running. I looked in AUTOEXEC.NCF and saw the BSTART command was still there - I don't know if it was ever removed or changed by either the uninstall or the upgrade/install. It did not add MGRSTART to the file. Btrieve is the transactional database interface, started by typing BSTART, while the relational database service allows access to the data through ODBC and OLE DB, and is started by typing MGRSTART. However, if you look at the mgrstart.ncf file, you'll notice that it also loads bstart.ncf. So, you'll only need to start one or the other, because with mgrstart, both are loaded anyhow. It should be noted, however, that typing MGRSTOP does not run BSTOP! So, even though mgrstart loads both, mgrstop only stops the relational services. I haven't tried it, but I'm assuming that simply running bstop will unload the relational services too, because they are dependents. I edited AUTOEXEC.NCF and changed the line from BSTART.NCF to MGRSTART.NCF. I checked the version of both btrieve.nlm, which was 8.00, and nwmkde.nlm, which was 8.00.114.000. At the server console, I typed clilcadm -i, and sure enough, our 10 user licenses showed up. All documentation you see for psql tells you to increase the MicroKernal
cache size to at least 20% of your total physical memory. This just
seems ridiculously high to me. We have 2GB in the server, and there's
no way it needs 400MB. I searched around a bit, and saw reports that
psql still has great performance when giving it only 20-30MB cache.
So, I edited my \\sys\system\BTI.CFG file and set the cache size
to 32MB (the install does NOT set the cache size automatically, at least
not when you upgrade from an existing psql installation - the stock
v2000i that comes installed with Netware sets it to 2MB.):
Next, I wanted to make sure we were had the most current psql updates. Apparently there is a v8 Security Feature Pack that brings the version up to 8.5, then Service Pack 2 that brings the version to 8.6, and finally Service Pack 3 that brings the version to 8.7. Each one of these are complete installation programs, so it does not appear that they have to be installed sequentially. I went to the updates page and downloaded the Service Pack 3 release for Netware. I ran the setup on the SP3, and it looked just like the normal installation. I remembered to do a mgrstop and bstop on the server first this time, just in case! The setup ran without a hitch, so I rebooted the server. Once again I checked the versions, and this time btrieve.nlm was version 8.70.12 and nwmkde.nlm was version 8.70.12.
First, I had to install a SCSI card for the tape drive. We had an Adaptec AVA-2906 sitting around, which is not supported for Netware, but I decided to give it a try. It's tough to find documentation about installing new hardware in Netware, so I decided to pop it in, boot up, and see what happened. I didn't watch it boot up, so if there's any sort of kudzu type thing that detects hardware, I missed it. I looked at what drivers are loaded for other Adapted cards, and I saw that the AHA-2940UW loads AHA2940.HAM. First, I checked for any sort of AVA modules by typing modules av*, but nothing showed up. I tried modules ah*, and saw that AHA2940.HAM was loaded! Hhhmmm.. I don't know if this was running before or not. The only other scsi card in the computer is an LSI Logic MegaRaid. Perhaps Netware had detected the card? Only one way to find out, and that was to install BrightStor ArcServe. You can install BrightStor either from the Netware Server or from a Windows client. I decided to do it from a Windows client so that I might have the option of installing the manager and other Windows components as the same time. To install from a Windows client, you must have at least Netware Client 4.83, with an account that has admin rights to the Netware Server, Supervisor Object Rights to the tree (in order to extend the NDS schema), and Supervisor Object rights to the container in which the server object resides. In addition, at least one supported SCSI adapter and any tape drives should already be in place. All of that was set, and I made sure I was ONLY logged into the NW6.5 server (we have two servers with the same name online right now, but on different trees and using different protocols). I inserted the disc and selected the first option, Install. From the next screen, I selecte Install BrightStor ARCserve Backup 9.0 for Netware Server and Options. At the next screen, I selected Install Server components. I clicked Next until I got to the screen that let me select the server on which to install, and I selected S3. I knew that because I was not logged into the old S3, it would have to prompt me for a username/password if it wanted to install on that one, which I could then cancel due to it being the wrong server! At the next screen, the path for installation became netware://Servers/S3/FileSystem/SYS/ARCserve. If you had an older version of BrightStor and/or Btrieve (pre 7.94), a screen would pop up at this point from which you could select to overwrite the old ArcServe application/database or upgrade Btrieve. I entered the license key at the next screen and clicked Next. At the next screen, I typed in the DNS name of the computer on which I would install the Backup Manager and CA Alert Service (a windows workstation). At the next screen, I left Install InoculateIT checked, even though I knew I probably wouldn't use it. This installed it at netware://Servers/S3/FileSystem/SYS/ARCserve/INOCULAN. It gave me that final review screen before installation, and I clicked Finish. After it copied a bunch of files, a password prompt came up so that it could create ARCserve Backup NDS objects. The account on that popup screen was the correct one for our NW6.5 server, so I typed in its password and clicked OK. The next screen lets you configure drivers. I highlighted NWPA Driver Interface (the description reads "Select this device driver to use with the direct SCSI attached device.") from the list of available drivers, clicked Add and then clicked OK. Installation complete! I checked "Launch the Device Configuration utility" and clicked Close. At the second Device Configuration screen, I checked to configure Stand-Alone Drive(s), since we have no libraries (we will just have a bunch of individual drives). You can also select File System Devices to use hard drive storage space as a device, which I might want to take advantage of at some point. But I will run the Device Configuration again to set that up when the time comes. I clicked Next. At this point, a window popped up that said "BrightStor ARCserve Backup is not running on the server. Please issue ASTART command on Netware console to start the backup engine." Ok, so I went to the Netware Console and typed ASTART. When I did so, it said that PFC.NLM reported a critical error. I flipped over to the Logger Screen, scrolled up, and found the following
error:
From the NLM Loader screen, I also saw the errors:
Just for the heck of it, I went back to the Device Configuration screen, clicked OK on the error, then clicked Next to try again. I still got the same error! Hhhmmm...I took a look at the \\sys\arcserve\PFC.INI file, and it appeared that the [MODULES570] section was a section that applied to Netware 5.7 servers! I didn't even know if there was a NW5.7, but we surely weren't running it. There were only module sections for NW4.11, NW5.0 and NW5.6 in that file! I searched around, and apparently we needed a new PFC.INI file. On
Ca's website, I saw a similar
answer for NW6.0:
I looked at the patches on their website, and decided to apply the patches to bring the version up to 9.01. First, I applied update QO45869 (whose readme is here) by extracting the patch, CAZIPXP -U QO45869.CAZ, stopping Arcserve by typing ASTOP, and running the executable, BAB901NE. The installation was quick - too quick. In fact, I'm not sure it did anything! I also noticed that it disconnected me from the server during the install. I typed ASTART, and it looked like we still had the same problem (and, it was still loading v9.00). I decided to reboot the server and try again, and still the same thing. This time I tried it from a different computer - namely, the one that I had originally installed AS from. It still looked way too quick, but I tried typing ASTART, and this time the version was 9.01! I watched it load, and everything looked fine! We no longer had that error. It also recognized the SCSI card and drive, even though it wasn't actually supported! Aww, yeah. It did, however, tell me "ERROR NO GROUPS REGISTERED: RUN DEVICE CONFIGURATION" ...of course. I decided to continue with the patches, this time installing v9.01 patch 1, QO53395. Same installation method as before, and went off without a hitch. I then installed QO53396, which is an update (or complete install) of BAOF (backup agent for open files), which I had not yet installed on the server (but I have it now, as of this update! Although, it would not let me use our keycode - it filled it in with the default live trial keycode). You have to install it to the local computer, then run it, select the server, and click install. To start it on the server, you must type LOAD OFA. (BFO?). I installed the remaining patches, QO57736 and QO64213, both of which simply involve overwriting certain files. This should make our installation completely up to date. At this point, I thought it would be a good idea to install the Netware Manager and Windows Agent from the CD, then run the updates again to get everything current on the managing workstation. I ran the setup, selected all components, and clicked Next. It then asked me for the Netware Discovery Server Name, the Discovery Server apparently being a service that listens on UDP port 41524 and discovers other backup servers on the network. According to Ca, "The Discovery Server is the server that will be the ARCserve Host server." I was a little bit iffy about putting anything in here, because we only plan on having one backup server on the network, but I filled in the name of our server, S3, anyhow. I tried running the first patch, QO53395, on the local install, but it always got the following error: Cannot install files from 'sbsUSA.mva': Hardware error. I always clicked "Continue Install," but this error (and another on the file sbs.mva) popped up several more times. Nonetheless, the update finished and stuff still ran. So perhaps I have half an update! Finally, I installed QO64541, which is a security fix for the Discovery Service on Windows systems. This updated the local managing workstation install, but it looks like it affected the open file agent as well, so to make sure, I did another install of BAOF from this workstation to the server after the patch was applied. When I typed ASTART again on the server, everything loaded fine until I got to TAPESVR.NLM, which game the following error:
Hhhmmm, this is new. In the two updates that had you manually replace files, TAPESVR.NLM was one of the drivers that was replaced. I searched around and found this kb article that stated that tapesvr now actively checks for conflicting CDM's, whereas it didn't before. Ok, so before, we probably would have run into trouble. I guess it's good that it checks! I did what it said. First, I typed UNLOAD NWTAPE.CDM. I checked AUTOEXEC.NCF, but there was no mention of the driver in there. I rebooted the server into DOS and took a look at STARTUP.NCF, but there was no mention there either. I decided to see what would happen when I booted up the server and tried starting Arcserve again. Same error! Where was nwtape.cdm being loaded from?? I grep'd the SYS drive, but turned up nothing. I searched around, and according to Veritas (yes, I know this product is not Veritas!),
I bet that's it. However, I read some places that simply unloading the .CDM file may still cause problems - it must never be loaded in the first place! I decided to reboot once again into DOS and simply rename it. It was located in C:\NWSERVER\DRIVERS. *** In fact, I ran into this same problem again later on, which I fixed by renaming NWASPI.CDM and DLTTAPE.CDM as well as NWTAPE.CDM. Make sure none of these are being loaded, by booting to DOS and renaming or deleting them from the Bootstrap partition. Later on, when I ran into this problem yet again, I had to rename the corresponding .DDI files as well. I tried booting up and typing ASTART, and this time everything loaded perfectly (that means no error about nwtape.cdm)! Time to run Device Configuration. I entered the server FQDN and the admin account/password, then reached the next screen that only let me select the options Tape/Optical Library or File System Device. I did not want to configure any of these, so I exited. Instead, I ran Manager, right-clicked on the drive, and selected Device Group Configuration. I clicked on New and created a group, then assigned the drive to it. Voila! In Server Admin -> Configuration, I chose NOT to purge the
database, since I still wanted a list of tapes/jobs for many years to
come, but turned on pruning to happen after 180 days. So, records
of tapes and jobs will always be there, but to restore anything older
than six months, the tape will have to be merged.
======================== Backup Agent for Open Files (BAOF) I decided to run a full backup of the server - this included all volumes, plus NDS. It ran fine, but the verification failed because on certain files, timestamps or file size had changed (mostly on .TDS files, such as VOLDATA.TDF), or the file was open and could not be backed up. I had not loaded BAOF, so I decided to do so. This needs to be loaded before anything is opened on the volumes, so I put it right at the beginning of AUTOEXEC.NCF, right after the time zone stuff (LOAD OFA.NLM). Then, I rebooted. Turned out OFA didn't load, or exited for whatever reason. Perhaps I was loading it too soon. Loading it later on, from the System Console, still caused some open files to be skipped! As it turns out, BAOF license does not come with the base Arcserve product! It is a standalone, and it is expensive. Netware keeps a file level snapshot on all its NSS volumes, so why do we need this?? Here is a Novell document abotu how the File-Level Snapshots work. *A BAOF whitepaper can be found here. *You can check the license on BAOF by typing LOAD OFA -D on the server. This will not actually load OFA, but it will create a file, SYS:\SYSTEM\BAOFDIAG.TXT that will hold the license info. *** I later decided to uninstall BAOF, since I sure as hell wasn't going to purchase it. To do so, first I stopped Arcserve by typing ASTOP. I then killed BAOF by typing UNLOAD OFA.NLM. I then deleted the BAOF directory, SYS:/SYSTEM/OFA, and deleted the SYS:/SYSTEM/OFA.NLM file. I rebooted, and voila!
========================= Creating Backup Administrator accounts First, you need to add a user to the AQ6_<server name> account. In ConsoleOne, click on Properties, Other tab, and expand User. I created a group called BackupUsers, to which I added accounts that will be Backup Admins. I added this to the User attribute (it may need to be added Operator as well, but for now, I'm playing it safe). I added the admin account to this group, but also kept the admin as a user/operator (redundancy!). There are also profiles in Arcserve. From the left-hand menu in Manager, select Utilities -> User Profile. You can highlight one of the default profiles (or create your own), and then select a user from the context menu on the right. However, you can not seem to be able to add groups using this method, so I did not do it. At first, I assumed that the Backup User group would need rights to read NDS as well, since they should be able to back that up. What sort of rights? Here is a good article (TEC268206) describing the rights you need in the filesystem to perform backups/restores (this does not talk about NDS). Apparently, by default, ANYONE can submit jobs. I created A NEW USER, WITH NO RIGHTS, and tried running a backup - first of NDS, then of one of the volumes. The jobs were submitted and ran just fine. Granted, when I looked at what was backed up, the jobs were pretty much empty; only the objects and files that the user had rights to were actually backed up. It troubled me, though, that the user was able to submit a job at all! Not only that, but the user could also see every directory and file on the system, whether they had access to it or not. It was obviously using Arcserve's built-in account to access the system, rather than the user's, and opening a rather large security hole. So, the user can backup, restore, and see ALL files on the system, regardless of either their Netware rights or the profile settings in Arcserve. Fun. I snooped around a little bit trying to find an explanation, and it appears that when a user starts Manager and connects to the server, he adopts the rights of the user that started Arcserve from the Console - namely, the Admin. This must be true for browsing, viewing the database/logs, and submitting jobs, at least. The user can still only backup and restore to locations or objects that he has rights to. I deduced this by removing all users (including Admin) from the Operator and User fields, as well as all trustees, for both the Cheyenne Queue (AQ6_S3) and Cheyenne User (AS_BACKUP_SERVER) objects. I also checked these objects' rights to other NDS objects, and they had none. They also did not seem to have any rights to most of the filesystem, including directories that showed up when browsing with Manager. I checked my unprivileged user's effective rights to these objects, and he had none. I changed the Admin password in Netware, in case that was stored somewhere and hence being used (after all, I did install Arcserve onto the server with Admin). Nada. Nothing. I could still see everything with my unprivileged runt. I tried a restore of both an NDS object (a user group in the same container as the server) and a directory off the base dir of one of our volumes, using a user with no rights. Neither was actually restored, and the job failed with an Incomplete. The log showed "E3208 Write error while restoring ..." for the directories on the file system, but a Successful for the NDS object, even though it wasn't actually restored. I then tried a backup of both NDS (I tried the entire tree) and a file system volume, using that same unprivileged user. According to the logs, the job was successful, and was verified correctly. When I went into Manager (using a privileged account) and checked out those sessions, the NDS session was completely empty (nothing was actually backed up) and the file system volume only backed up the one file that the user had access to, with the rest of the directories not showing up at all (even though they showed up in Manager when the user selected which files to backup). When I tried backup up one individual object in NDS with the unprivileged account, it said that it was successful. In addition, when looking at the session on the backup tapes, it also said that the object was there! However, when I tried restoring it using the privileged account, it again said that it was successful, but nothing was actually restored to the NDS. I tried restoring it with the unprivileged account as well, just for shits and giggles (and since it was the same account that made the supposed backup), but obviously that didn't work either. And yes, I did try this same restoration on a backup made with a privileged account, and it did backup AND restore just fine, with the objects actually appearing back in NDS. :) At any rate, ignoring the huge security hole for now, I decided to
give my BackupAdmins group the proper rights. According to the document
I mentioned a couple paragraphs up:
Giving the account group the Supervisor Object right to the server
will also give it Supervisor File System rights to all the volumes,
which will let it restore any files to anywhere, give it access to see
all those files in the backup jobs, and let it restore the original
owner, trustees and file system rights. So, that's what I did - I gave
the account group the Supervisor Object rights to the Server object,
as well as to AS_BACKUP_SERVER and AQ6_S3. At this point, with the account having Supervisor Object rights to the server, and hence all volumes on the File System, I tried a backup and restore of a volume and also an NDS object. This time, all dirs/files on the volume showed up in the backup, and could be restored, with correct permissions/owners, both by that user and by another admin user. The NDS object backup still pretended like it was successful, and the object showed up in the session, but again, it could not be restored by either that user or an admin user. So at this point, I had an account that could fully do file system backups/restores. *** It should be noted that there seems to be a bug in ArcServe (we are running v9.01 with all patches as of January, 2006). When restoring directories on a given volume, at least in the root dir (and excluding any files in root, such as voldata.tdf), it assigns all of the correct ownerships/permissions, except on the very last directory, to which it makes the account used to perform the restore the NEW OWNER of that directory and all of its contents!!! I thought this might be a rights issue, so I tried a restoration with my admin account, and that assigned the correct owner to all directories off of root. All files and subdirs under those directories, however, were assigned that admin account as the owner!! Hhhmmmm... Also, Keep in mind that when an NDS object, such as a user, is deleted and then restored from a backup, the file system trustee rights are not restored. This is because the trustee rights are stored at the actual directory level in the file system and not in NDS. When a user is deleted, any trustee rights that he held are deleted as well (and, they were never stored on the backup of NDS). It is possible to do a restore of volumes with a "Restore Trustees Only" option selected, which will only restore the trustee settings and also the directory structure.
After I installed eDirectory 8.8 and Security Services 2.01, I could
no longer submit backup jobs that included a backup of NDS. I would
always get the following errors (in the job log, SYS:\ARCserve\JBSTATUS.LOG\*.log,
or in SYS:\ARCserve\ARCH$SVR.log):
The account that I tried to submit the backup job with would then be locked out via Intruder Lockout for too many failed login attempts. I tried putting in the full context for the user, or writing out the context a million ways (both typeful, such as .CN=admin.O=DataStat or typeless such as .admin.DataStat), but nothing seemed to work. The accounts I tried had Supervisor Object rights to [root], there were no IRF's blocking this, and were security equivalent to the server object. SMDR.NLM was also loaded (this is Novell's Storage Management System, which allows you to back up namespace information). I'm fairly certain that "User SMS for backup, restore and copy jobs" was selected when installing ArcServe. There were no files in the SYS:\SYSTEM\TSA directory (if no more files can be created in this directory, TSA will be unable to login to the server, and you will get the same error). I did a lot of reading, and many people apparently solved this problem by moving their server's license certificate to the root container. Our server is only one container deep (in an Organization rather than an Organizational Unit), and you can't move it to [root], so I could not attempt to solve it that way. I also tried recreating the SMS configuration and objects by following this document, but ended up getting the same error when I was done. ***Note that with SMDR 6.50c and later, the SMDR Group Object and SMDR RPC Objects are no longer used, and therefor not created when you run SMDR NEW. Also note that you must run TSAFS to recreate the \ETC\SYS\TSA.CFG file. When I put in the full typeless context of the admin, which included the root (.admin.DataStat.DROOT), the account did get locked out, and it did successfully backup the file system. However, it could still not log into TSANDS, and the NDS portion of the backup failed. Progess at least!
ALBUILD: Failed to open alert Initialization file. This is loaded after ASTART and is part of Inoculan. Auto-loaded by either ICORE.NLM or INOCUCMD.NLM. ALSTART.NCF? Delete detail files from Server Admin -> Configuration? Please add MAC name space to restore S3/COW:\ICON_~1 after backup??? Finally, everything up to date and loading correctly, so once again I ran the Device Configuration from the managing workstation.
I would always get the following error from the ARCserve Message Screen right after everything else loaded (when running ASTART): Failed to send Alerts to [asmanager.datastat.com]. As it turned out, I simply needed to add that computer name/IP address to the HOSTS file (SYS:/etc/hosts). After that, stop and restart ARCserve. Voila! Now, I get the following message: Alerts will be sent to [asmanager.datastat.com], so I know that computer is reachable. Alerts have to be set up manually. From Manager, hit Server Admin
from the left-hand menu, then select Admin -> Configuration. Under
the Alert tab, enter the text that should cause an alert. I added
the following pieces of text:
Then, select Alert from the left-hand menu in Manager. This will open the Alert Manager UI. I got the following error the first time: "The Alert Account information for machine CONSOLE1 is incorrect. Please enter an Administrator equivalent account that has the right to Logon as a Service. [2]." I entered the Administrator account.s *** When you select "Backup," fill in your username/password and check the "remember the security information" checkbox, it stores this information in a file called BrightStorMgr.dat in your Application Data directory. For me, this was C:\Documents and Settings\Administrator\Application Data\BrightStorMgr.dat. It is ok to delete this file - it is simply recreated when you run Manager again, but of course the saved username/password info will be gone.
AuthenticationBrok.1505 java.lang.NullPointerException Option [IncludeNCFFiles] is DISABLED BrightStor ARCserve server has valid job queue and queue user
|