This discussion has been locked.

You can no longer post new replies to this discussion. If you have a question you can start a new discussion

SQL backup failures to network share

Hello,

We recently stood up a new backup enviornment using HP StoreOnce (Deduplication applicance) as the backup target. The issue we're currently running into is when we perform a SQL backup using LiteSpeed (not compressed) to the CIFS share (\\hpeso\SQLBackups) on HP StoreOnce we get the following error:

- test backing up a 14GB database

Msg 61700, Level 16, State 1, Line 46

Backup of database "TestDatabase" failed.

Error saving data to file. Thread: 0

BACKUP DATABASE is terminating abnormally.

A nonrecoverable I/O error occurred on file "VDI_730BE351-F308-4C41-8F

C7-B6434FD61EF6_0:" 995(The I/O operation has been aborted because of either a thread exit or an application request.).

Write on "VDI_730BE351-F308-4C41-8FC7-B6434FD61EF6_0" failed: 4(The system cannot open the file.)

And when it's compressed by LiteSpeed we get a different error writing to the same backup target:

BackupIoRequest::ReportIoError: write failure on backup device 'VDI_9F406D37-986D-4A49-AA3C-844801FE9B6C_7'. Operating system error 4(The system cannot open the file.).

Is there anything I could do on the LiteSpeed configuration side in hopes to resolve this issue?

Any help in resolving this would be greatly appreciated!

0 Alexander over 6 years ago

Hello,

Accroding to the error you have check access for Sql Server account to the new storage (folder).
To get further investigation please open a support case.

Alexander.
Cancel
Up 0 Down

Cancel
0 MihirP1129 over 6 years ago

Thanks for the response Alexander.

The SQL server account already has access to the share. I went ahead and changed the permission on the share to allow everyone with full control (to rule out access issues) then ran the backup job again but it failed.

CPU Seconds: 11.22
Environment: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz
CPUs: 4 logical, 4 cores, 4 physical packages.

Msg 60404, Level 16, State 1, Line 0
Backup of database "VC" failed.
Error saving data to file. Thread: 0

BACKUP DATABASE is terminating abnormally.
A nonrecoverable I/O error occurred on file "VDI_DF332AF0-703A-41E5-B04E-4C736EC52E4D_0:" 995(The I/O operation has been aborted because of either a thread exit or an application request.).
Write on "VDI_DF332AF0-703A-41E5-B04E-4C736EC52E4D_0" failed: 4(The system cannot open the file.)

Not a valid LiteSpeed™ for SQL Server® format file (Type is: 1).
Select a valid LiteSpeed™ for SQL Server® file for this restore operation.

However, if I do a SQL native backup by going to the database --> tasks --> backup and specify the location \\hpeso\SQLBackups\testdb-full.bak, it works just fine!
Cancel
Up 0 Down

Cancel
0 Alexander over 6 years ago

Hello,

The error shows that OS could not open file for writing.

Please open a support case. Support should gather more information about your configuration.

From current ideas: You can try to connect the storage as a drive to the server, then check that storage is allowing all kind of writing types, i.e. overlapped writing should be allowed.

Thank you.
Cancel
Up 0 Down

Cancel
0 ben over 6 years ago

In addition to the steps Alexander mentioned. I would also check that your Litespeed backup script uses the below parameter.

,@init = 1

If you are trying to append a Litespeed backup to a native backup ( as may happen when you run a native backup test and then a Litespeed backup test using the same backup file name with @init = 0 in the Litespeed script ) then you would get the same error you are seeing.

EXAMPLE ERROR:

Not a valid LiteSpeed™ for SQL Server® format file (Type is: 1).

If this does not help then, as Alexander mentioned, please open a support case, so we can review a Litespeed log.

Hope that helps,
-Ben-
Cancel
Up 0 Down

Cancel
0 MihirP1129 over 6 years ago

After countless testing, the issue was related to encryption at rest being enabled on the storage target. Not sure, why LiteSpeed would bomb the backup job because of this (maybe, because I had @verify = 1?? and it can't verify the restore process of the encrypted data). But, once we disabled the encryption at the storage, the backup succeeded. I have a case open with support to find out why is this the case as we would like to have encryption at the storage level. Funny thing is the sql native backup jobs are successful on the encrypted share, but litespeed backup jobs are not.
Cancel
Up 0 Down

Cancel
0 Alexander over 6 years ago

Hello,

LiteSpeed updates the backup file header after writing all backup data. Is it possible that the backup process (SQL Server) has no rights/certificates/keys to read/re-open just written file (to verify it after backup) or move pointer inside the file at the storage since it was encrypted by the storage?

Alexander.
Cancel
Up 0 Down

Cancel
0 MihirP1129 over 6 years ago

That could very well be the case Alexander. I have a case open with HP on how the storage level encryption works and have sent the Support bundle logs over to LiteSpeed. I'm just hoping it's a simple setting I could make on LiteSpeed side to get this to work on the encrypted share.
Cancel
Up 0 Down

Cancel
0 johnwalker5025 over 6 years ago

Either the backup is damaged or the network returned incorrect data for the backup. see here: sqltechtips.blogspot.com/.../database-terminating-abnormally.html
Cancel
Up 0 Down

Cancel