This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Rapid Recovery Backup Volume Size 64 TB

Hello everyone!

 

Agent size volume E:\ 64 TB and used 50 TB it's backup failed with error "There was a problem with the VSS subsystem" and in event viewer error : VSS VolSnap Error 5 on agent. I'm not sure about memory on client or capacity for vss snapshot insufficient or any issue ?

 

 

Rapid Recovery 6.0.2

Client run on VMware ESXi : Windows 2012 R2 : Memmory 24 GB 

 

Thank you,

Parents
  • Hi Keerati:
    In plain English, "VSS VolSnap Error 5" means that there were insufficient resources allocated for the VSS snap job.

    I assumed that you have used VSSAdmin to inspect the state of your VSS writers and they are OK. If there are some issues, please refer to the following KB for info as how to reset VSS writers without a reboot.

    https://support.software.dell.com/appassure/kb/129774?cmpid=soc:appassure:communties:01

    Once we took this out of the equation, there are a few possible causes for the error you have encountered.

    The first that comes to mind is related to the size of your volume as  the limitation of the Volsnap driver is 64TB (while the limitation of the NTFS file system is 256TB). If it is the case, slightly shrinking the volume may fix the issue.


    Another possible issue may be related to the space taken by the VSS snapshots. There may be a few older VSS snapshots taking all the space allocated for VSS snapshots. There are two steps to fix this.

    1. use diskshadow to delete any previous vss snapshot

    2. make sure that there are no shadows enabled on the volume and that the shadows size limitations are removed

     At last, there may be some memory management issues as well. You can monitor the memory variations in Performance monitor. I am hesitant to advise you trying the fix below as it may or may not address this issue and the outcome may be harmful. In case you would like to poke around, strictly for information purposes, the key is

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management

    Add the following registry value:

    Value name: PoolUsageMaximum
    Data type: REG_DWORD
    Radix: Decimal
    Value data: 60

    Setting the value at 60 informs the Memory Manager to start the trimming process at 60 percent of PagedPoolMax rather than the default setting of 80 percent. If a threshold of 60 percent is not enough to handle spikes in activity, reduce this setting to 50 percent or 40 percent.

    Change the following registry value:

    Value name: PagedPoolSize
    Data type: REG_DWORD
    Radix: Hex
    Value data: 0xFFFFFFFF

    Setting PagedPoolSize to 0xFFFFFFFF allocates the maximum paged pool in lieu of other resources to the computer.

    Please note that I have added the "Registry usage disclaimer" as I consider the info above as interesting technical information but I would not use it in any production machine.

    On the troubleshooting side, you can always attempt creating a snapshot and get some more insight as of the causes by using diskshadow. Please see the example below. As you see, the shadow is removed after its creation (the "end backup" command). 

    PS C:\> diskshadow
    Microsoft DiskShadow version 1.0
    Copyright (C) 2013 Microsoft Corporation
    On computer:  W2012R2-TUDOR-0,  02-Nov-16 9:08:38 AM

    DISKSHADOW> SET VERBOSE on
    DISKSHADOW> set context volatile
    DISKSHADOW> add volume R:
    DISKSHADOW> begin backup
    DISKSHADOW> create
    COM call "lvssObject4->GetRootAndLogicalPrefixPaths" failed.
    Excluding writer "Shadow Copy Optimization Writer", because all of its components have been excluded.
    Excluding writer "BITS Writer", because all of its components have been excluded.
    Component "\TasksStore" from writer "Task Scheduler Writer" is excluded from backup,
    because it requires volume C:\ which is not in the shadow copy set.
    Component "\WriterMetadataStore" from writer "VSS Metadata Store Writer" is excluded from backup,
    because it requires volume C:\ which is not in the shadow copy set.
    Component "\PerformanceCounters" from writer "Performance Counters Writer" is excluded from backup,
    because it requires volume C:\ which is not in the shadow copy set.
    Component "\System Files" from writer "System Writer" is excluded from backup,
    because it requires volume C:\ which is not in the shadow copy set.
    Component "\Win32 Services Files" from writer "System Writer" is excluded from backup,
    because it requires volume C:\ which is not in the shadow copy set.
    Component "\BCD\BCD" from writer "ASR Writer" is excluded from backup
    because it requires a volume that cannot be found on this system.
    Component "\IISCONFIG" from writer "IIS Config Writer" is excluded from backup,
    because it requires volume C:\ which is not in the shadow copy set.
    Component "\Registry" from writer "Registry Writer" is excluded from backup,
    because it requires volume C:\ which is not in the shadow copy set.
    Component "\COM+ REGDB" from writer "COM+ REGDB Writer" is excluded from backup,
    because it requires volume C:\ which is not in the shadow copy set.
    Component "\WMI" from writer "WMI Writer" is excluded from backup,
    because it requires volume C:\ which is not in the shadow copy set.
    The writer "Task Scheduler Writer" is now entirely excluded from the backup,
    because it does not contain any components that can be included.
    The writer "VSS Metadata Store Writer" is now entirely excluded from the backup,
    because it does not contain any components that can be included.
    The writer "Performance Counters Writer" is now entirely excluded from the backup,
    because it does not contain any components that can be included.
    The writer "System Writer" is now entirely excluded from the backup,
    because it does not contain any components that can be included.
    The writer "IIS Config Writer" is now entirely excluded from the backup,
    because it does not contain any components that can be included.
    The writer "Registry Writer" is now entirely excluded from the backup,
    because it does not contain any components that can be included.
    The writer "COM+ REGDB Writer" is now entirely excluded from the backup,
    because it does not contain any components that can be included.
    The writer "WMI Writer" is now entirely excluded from the backup,
    because it does not contain any components that can be included.
    * Including writer "ASR Writer":
            + Adding component: \ASR\ASR
            + Adding component: \Volumes\Volume{c55e4f41-ef86-4151-aad0-450cc3b76f20}
            + Adding component: \Volumes\Volume{b9b22ef2-9ae9-11e6-80c6-00155d4b0c47}
            + Adding component: \Volumes\Volume{143a65bb-5208-49f7-a62d-fb66b2f22ca2}
            + Adding component: \Volumes\Volume{c826ff1f-a98a-4c31-91ae-0c2160c16541}
            + Adding component: \Volumes\Volume{ce3ee9d4-8959-423b-b0c4-d45966759866}
            + Adding component: \Disks\harddisk2
            + Adding component: \Disks\harddisk0
            + Adding component: \Disks\harddisk3
            + Adding component: \Disks\harddisk1
    Alias VSS_SHADOW_1 for shadow ID {475d85f9-4d74-42cf-ab88-bba125eca2eb} set as environment variable.
    Alias VSS_SHADOW_SET for shadow set ID {fe096cc9-5031-40ab-a4c7-f6304bb48685} set as environment variable.
    Inserted file Manifest.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file BCDocument.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM0.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM1.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM2.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM3.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM4.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM5.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM6.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM7.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM8.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM9.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM10.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file Dis40FC.tmp into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Querying all shadow copies with the shadow copy set ID {fe096cc9-5031-40ab-a4c7-f6304bb48685}
            * Shadow copy ID = {475d85f9-4d74-42cf-ab88-bba125eca2eb}               %VSS_SHADOW_1%
                    - Shadow copy set: {fe096cc9-5031-40ab-a4c7-f6304bb48685}       %VSS_SHADOW_SET%
                    - Original count of shadow copies = 1
                    - Original volume name: \\?\Volume{ce3ee9d4-8959-423b-b0c4-d45966759866}\ [R:\]
                    - Creation time: 02-Nov-16 9:09:42 AM
                    - Shadow copy device name: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1
                    - Originating machine: w2012r2-Tudor-0.BadRobot.net
                    - Service machine: w2012r2-Tudor-0.BadRobot.net
                    - Not exposed
                    - Provider ID: {b5946137-7b9f-4925-af80-51abd60b20d5}
                    - Attributes:  Auto_Release Differential
    Number of shadow copies listed: 1
    DISKSHADOW> end backup
    DISKSHADOW> list shadows all
    Querying all shadow copies on the computer ...
    No shadow copies found in system.
    DISKSHADOW>
     
    Hope that this helps.
     
    Windows Registry Disclaimer:
    Quest does not provide support for problems that arise from improper modification of the registry. The Windows registry contains information critical to your computer and applications. Make sure you back up the registry before modifying it. For more information on the Windows Registry Editor and how to back up and restore it, refer to Microsoft Article ID 256986 “Description of the Microsoft Windows registry” at Microsoft Support.
Reply
  • Hi Keerati:
    In plain English, "VSS VolSnap Error 5" means that there were insufficient resources allocated for the VSS snap job.

    I assumed that you have used VSSAdmin to inspect the state of your VSS writers and they are OK. If there are some issues, please refer to the following KB for info as how to reset VSS writers without a reboot.

    https://support.software.dell.com/appassure/kb/129774?cmpid=soc:appassure:communties:01

    Once we took this out of the equation, there are a few possible causes for the error you have encountered.

    The first that comes to mind is related to the size of your volume as  the limitation of the Volsnap driver is 64TB (while the limitation of the NTFS file system is 256TB). If it is the case, slightly shrinking the volume may fix the issue.


    Another possible issue may be related to the space taken by the VSS snapshots. There may be a few older VSS snapshots taking all the space allocated for VSS snapshots. There are two steps to fix this.

    1. use diskshadow to delete any previous vss snapshot

    2. make sure that there are no shadows enabled on the volume and that the shadows size limitations are removed

     At last, there may be some memory management issues as well. You can monitor the memory variations in Performance monitor. I am hesitant to advise you trying the fix below as it may or may not address this issue and the outcome may be harmful. In case you would like to poke around, strictly for information purposes, the key is

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management

    Add the following registry value:

    Value name: PoolUsageMaximum
    Data type: REG_DWORD
    Radix: Decimal
    Value data: 60

    Setting the value at 60 informs the Memory Manager to start the trimming process at 60 percent of PagedPoolMax rather than the default setting of 80 percent. If a threshold of 60 percent is not enough to handle spikes in activity, reduce this setting to 50 percent or 40 percent.

    Change the following registry value:

    Value name: PagedPoolSize
    Data type: REG_DWORD
    Radix: Hex
    Value data: 0xFFFFFFFF

    Setting PagedPoolSize to 0xFFFFFFFF allocates the maximum paged pool in lieu of other resources to the computer.

    Please note that I have added the "Registry usage disclaimer" as I consider the info above as interesting technical information but I would not use it in any production machine.

    On the troubleshooting side, you can always attempt creating a snapshot and get some more insight as of the causes by using diskshadow. Please see the example below. As you see, the shadow is removed after its creation (the "end backup" command). 

    PS C:\> diskshadow
    Microsoft DiskShadow version 1.0
    Copyright (C) 2013 Microsoft Corporation
    On computer:  W2012R2-TUDOR-0,  02-Nov-16 9:08:38 AM

    DISKSHADOW> SET VERBOSE on
    DISKSHADOW> set context volatile
    DISKSHADOW> add volume R:
    DISKSHADOW> begin backup
    DISKSHADOW> create
    COM call "lvssObject4->GetRootAndLogicalPrefixPaths" failed.
    Excluding writer "Shadow Copy Optimization Writer", because all of its components have been excluded.
    Excluding writer "BITS Writer", because all of its components have been excluded.
    Component "\TasksStore" from writer "Task Scheduler Writer" is excluded from backup,
    because it requires volume C:\ which is not in the shadow copy set.
    Component "\WriterMetadataStore" from writer "VSS Metadata Store Writer" is excluded from backup,
    because it requires volume C:\ which is not in the shadow copy set.
    Component "\PerformanceCounters" from writer "Performance Counters Writer" is excluded from backup,
    because it requires volume C:\ which is not in the shadow copy set.
    Component "\System Files" from writer "System Writer" is excluded from backup,
    because it requires volume C:\ which is not in the shadow copy set.
    Component "\Win32 Services Files" from writer "System Writer" is excluded from backup,
    because it requires volume C:\ which is not in the shadow copy set.
    Component "\BCD\BCD" from writer "ASR Writer" is excluded from backup
    because it requires a volume that cannot be found on this system.
    Component "\IISCONFIG" from writer "IIS Config Writer" is excluded from backup,
    because it requires volume C:\ which is not in the shadow copy set.
    Component "\Registry" from writer "Registry Writer" is excluded from backup,
    because it requires volume C:\ which is not in the shadow copy set.
    Component "\COM+ REGDB" from writer "COM+ REGDB Writer" is excluded from backup,
    because it requires volume C:\ which is not in the shadow copy set.
    Component "\WMI" from writer "WMI Writer" is excluded from backup,
    because it requires volume C:\ which is not in the shadow copy set.
    The writer "Task Scheduler Writer" is now entirely excluded from the backup,
    because it does not contain any components that can be included.
    The writer "VSS Metadata Store Writer" is now entirely excluded from the backup,
    because it does not contain any components that can be included.
    The writer "Performance Counters Writer" is now entirely excluded from the backup,
    because it does not contain any components that can be included.
    The writer "System Writer" is now entirely excluded from the backup,
    because it does not contain any components that can be included.
    The writer "IIS Config Writer" is now entirely excluded from the backup,
    because it does not contain any components that can be included.
    The writer "Registry Writer" is now entirely excluded from the backup,
    because it does not contain any components that can be included.
    The writer "COM+ REGDB Writer" is now entirely excluded from the backup,
    because it does not contain any components that can be included.
    The writer "WMI Writer" is now entirely excluded from the backup,
    because it does not contain any components that can be included.
    * Including writer "ASR Writer":
            + Adding component: \ASR\ASR
            + Adding component: \Volumes\Volume{c55e4f41-ef86-4151-aad0-450cc3b76f20}
            + Adding component: \Volumes\Volume{b9b22ef2-9ae9-11e6-80c6-00155d4b0c47}
            + Adding component: \Volumes\Volume{143a65bb-5208-49f7-a62d-fb66b2f22ca2}
            + Adding component: \Volumes\Volume{c826ff1f-a98a-4c31-91ae-0c2160c16541}
            + Adding component: \Volumes\Volume{ce3ee9d4-8959-423b-b0c4-d45966759866}
            + Adding component: \Disks\harddisk2
            + Adding component: \Disks\harddisk0
            + Adding component: \Disks\harddisk3
            + Adding component: \Disks\harddisk1
    Alias VSS_SHADOW_1 for shadow ID {475d85f9-4d74-42cf-ab88-bba125eca2eb} set as environment variable.
    Alias VSS_SHADOW_SET for shadow set ID {fe096cc9-5031-40ab-a4c7-f6304bb48685} set as environment variable.
    Inserted file Manifest.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file BCDocument.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM0.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM1.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM2.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM3.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM4.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM5.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM6.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM7.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM8.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM9.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file WM10.xml into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Inserted file Dis40FC.tmp into .cab file 09-02-Nov-16-09_-AM-_W2012R2-TUDOR-0.cab
    Querying all shadow copies with the shadow copy set ID {fe096cc9-5031-40ab-a4c7-f6304bb48685}
            * Shadow copy ID = {475d85f9-4d74-42cf-ab88-bba125eca2eb}               %VSS_SHADOW_1%
                    - Shadow copy set: {fe096cc9-5031-40ab-a4c7-f6304bb48685}       %VSS_SHADOW_SET%
                    - Original count of shadow copies = 1
                    - Original volume name: \\?\Volume{ce3ee9d4-8959-423b-b0c4-d45966759866}\ [R:\]
                    - Creation time: 02-Nov-16 9:09:42 AM
                    - Shadow copy device name: \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1
                    - Originating machine: w2012r2-Tudor-0.BadRobot.net
                    - Service machine: w2012r2-Tudor-0.BadRobot.net
                    - Not exposed
                    - Provider ID: {b5946137-7b9f-4925-af80-51abd60b20d5}
                    - Attributes:  Auto_Release Differential
    Number of shadow copies listed: 1
    DISKSHADOW> end backup
    DISKSHADOW> list shadows all
    Querying all shadow copies on the computer ...
    No shadow copies found in system.
    DISKSHADOW>
     
    Hope that this helps.
     
    Windows Registry Disclaimer:
    Quest does not provide support for problems that arise from improper modification of the registry. The Windows registry contains information critical to your computer and applications. Make sure you back up the registry before modifying it. For more information on the Windows Registry Editor and how to back up and restore it, refer to Microsoft Article ID 256986 “Description of the Microsoft Windows registry” at Microsoft Support.
Children
No Data