Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UDF File System Layout #6

Open
fengjiannan2010 opened this issue Apr 30, 2024 · 4 comments
Open

UDF File System Layout #6

fengjiannan2010 opened this issue Apr 30, 2024 · 4 comments
Labels
question Further information is requested

Comments

@fengjiannan2010
Copy link

When burning a 100GB file onto a disc under normal circumstances, I need to establish the file system layout, specifying the starting address, file size, file name, and file attributes for the file to be burned. Now, I wish to be able to write the file in fragments, with each fragment being 10GB, allowing for the possibility of fragment write failures to be rewritten during the writing process. The final file layout will need to be modified into a UDF linked list, specifying all segments of the file as well as the starting address and size of each fragment.

udf_file_system_layout

To meet this requirement, you can utilize the Universal Disk Format (UDF) file system to manage data on the optical disc. UDF is a file system used for optical discs and other removable storage media, offering support for high-capacity storage and the ability to dynamically add/remove files.

In the UDF file system, files are organized into a series of descriptors and data areas. For large files, UDF supports segmented storage, facilitating handling of large files and allowing segmented writing. You can employ UDF's linked list structure to track file segments and their associated information.

Below is a simplified layout example of a UDF file system:

  • Anchor Volume Descriptor Pointer (AVDP): Identifies the starting position of the file system.
  • Volume Recognition Sequence (VRS): Used to recognize the UDF file system.
  • Logical Volume Descriptor (LVD): Contains descriptors with overall information about the file system.
  • File Set Descriptor (FSD): Describes the root directory of the file set.
  • Partition Descriptor (PD): Contains information about partitions if the disc is divided into multiple partitions.
  • File Entry Descriptor (FED): Describes file metadata such as name, size, attributes, etc.
  • Allocation Descriptor (AD): Describes the starting address and size of file data segments.

For your requirements, here's a modification plan for the UDF file system layout:

  1. Creating large files in the UDF file system: Segment the 100GB file and use AD descriptors to track the location and size of each segment.
  2. Allowing rewrites after segment writing failures: If a segment writing fails, mark the status in the corresponding AD descriptor and update it during rewrite.
  3. Tracking segments using a linked list structure: Create a linked list structure for each file, where each node represents a segment. Each node contains an AD descriptor and a pointer to the next node. This way, when you need to add a new segment to the file, simply update the last node in the list by appending the AD descriptor of the new segment to the end of the list.

When implementing the UDF file system, consider the UDF specifications and related file system operations. You may need to use specialized UDF file system libraries or tools to build and manage the UDF file system.

I want to write large files in this way and store them in UDF partitions. I read large files through network streaming and write them using BlockDevice. Due to network reasons, the data is written abnormally and the entire disc becomes unusable. I need to use a blank disc to rewrite. If I write according to file fragments and record the fragment information (address, size, file name), and finally organize it into a UDF file system, can PrimoBurner support the UDF partition storage interface?

@vkantchev vkantchev added the question Further information is requested label Apr 30, 2024
@vkantchev
Copy link
Contributor

This is currently supported using DataFile.UdfFileProps.Extents. You can describe the file fragments via UdfExtent objects and add those objects to the DataFile.UdfFileProps.Extents list. BlockDevice will take into account the fragments.

@vkantchev
Copy link
Contributor

Here is an example:

        private static DataFile CreateFileSystemTree()
        {
            DataFile oRoot = Library.CreateDataFile();

            oRoot.IsDirectory = true;
            oRoot.FilePath = "\\";
            oRoot.LongFilename = "\\";

            for (int i = 0; i < _sourceFiles.Count; i++)
            {
                TFile file  = _sourceFiles[i];

                DataFile df = Library.CreateDataFile();

                    // it is a file
                    df.IsDirectory = false;

                    // filename long and short
                    df.LongFilename = Path.GetFileName(file.Path);
                    df.ShortFilename = "";

                    df.DataSource = DataSourceType.Disc;	      // it is already on the cd/dvd
                    df.DiscAddress = file.Extents[0].DiscAddress; // set the disc address from the first extent
                    df.FileSize = file.Extents[0].DiscSize;		  // and the size

                    if (file.Extents.Count > 1)
                    {
                        // We have multiple extents. We need to add them to DataFile.UdfFileProps.Extents
                        for (int j = 1; j < file.Extents.Count; j++)
                        {
                            UdfExtent ext = Library.CreateUdfExtent();

                                ext.Address = file.Extents[j].DiscAddress; // set the disc address
                                ext.Length = file.Extents[j].DiscSize; // and the size
                                
                                df.UdfFileProps.Extents.Add(ext);

                            ext.Dispose();
                        }
                    }

                    oRoot.Children.Add(df);

                // Must dispose the object
                df.Dispose();
            }

            return oRoot;
        }

@vkantchev
Copy link
Contributor

Full code (for Windows):

using System;
using System.IO;
using System.Collections.Generic;
using System.Diagnostics;
using System.Text;
using System.Runtime.InteropServices;
using PrimoSoftware.Burner;

namespace BlockDevice2
{
    // Structure to keep the file extent information
    public struct TExtent
    {
        // address on the CD/DVD
        public int DiscAddress;

        // size on the CD/DVD (in blocks)
        public long DiscSize;
    };

    // Structure to keep the file information for BlockDevice
    public class TFile
    {
        // full path to the file
        public string Path;

        // file extents
        public List<TExtent> Extents = new List<TExtent>();
    };

    enum BurnOption
    {
        Unknown	= 0,
        Write,		// write
        ReadDiscID,	// read
        Erase		// erase
    }

    class MainClass
    {
        // Size must be aligned to 16 blocks
        private const int BLOCKS_PER_WRITE  = 10 * 16;

        /////////////////////////////////////////////
        // Command line handlers 
        //
        static void Usage()
        {
            Console.Write("BlockDevice [-e] [-i <sourcefile>] [-i <sourcefile>] [-i <sourcefile>]\n");
            Console.Write("    -i sourcefile = file to burn, multiple files can be specified.\n");
            Console.Write("    -e            = erase RW disc. \n");
            Console.Write("    -d            = read and display temporary disc ID. \n");
        }

        static int ParseCommandLine(string[] args)
        {
            int		i = 0 ;
            string	sCommand = "";	

            for (i=0; i < args.Length; i++)
            {
                sCommand = args[i];

                //Input file
                if (sCommand == "-i")
                {
                    i++;
                    if (i == args.Length)
                    {
                        Usage();
                        return -1;
                    }

                    TFile f = new TFile();
                    f.Path = args[i];

                    _sourceFiles.Add(f);

                    continue;
                }

                //Erase
                if (sCommand == "-e")
                {
                    if (_burnOption != BurnOption.Unknown)
                    {
                        Usage();
                        return -1;
                    }

                    _burnOption = BurnOption.Erase;
                    continue;
                }

                // Read Disc ID
                if (sCommand == "-d")
                {
                    if (_burnOption != BurnOption.Unknown)
                    {
                        Usage();
                        return -1;
                    }

                    _burnOption = BurnOption.ReadDiscID;
                    continue;
                }
            }

            if (_burnOption == BurnOption.Unknown)
                _burnOption = BurnOption.Write;

            if ((BurnOption.Erase != _burnOption) && (BurnOption.ReadDiscID != _burnOption) && (_sourceFiles.Count == 0))
            {
                Usage();
                return -1;
            }

            return 0;
        }

        private static Device SelectDevice(Engine engine)
        {
            int ch;
            int nDevice;

            // A variable to keep the return value
            Device device = null;

            // Get a device enumerator object. At the time of the creation it will enumerate fileSize CD and DVD Writers installed in the system
            DeviceEnumerator Enum = engine.GetDevices();
    
            // Get the number of available devices
            int nCount = Enum.Count;

            // If nCount is 0 most likely there are not any CD/DVD writer drives installed in the system 
            if (0 == nCount) 
            {
                Console.WriteLine("No devices available.");

                // Release the enumrator to free any allocated resources
                Enum.Dispose();
                return null;
            }

            // Now ask the user to choose a device
            do
            {
                Console.WriteLine("Select a device:");

                // Loop through fileSize the devices and show their name and description
                for (int i = 0; i < nCount; i++) 
                {
                    // Get device instance from the enumerator
                    device = Enum.GetItem(i);

                    // GetItem may return null if the device was locked
                    if (null != device)
                    {
                        // Get device description
                        string strName;
                        strName = device.Description;

                        // When showing the devices, show also the drive letter
                        Console.WriteLine("  ({0}:\\) {1}:", device.DriveLetter, strName);

                        device.Dispose();
                    }
                }

                Console.WriteLine("Enter drive letter:");
                ch = getch();

                nDevice = Library.GetCDROMIndexFromLetter(Convert.ToChar(ch));
            } 
            while (nDevice < 0 || nDevice > nCount - 1);

            device = Enum.GetItem(nDevice);
            if (null == device)
            {
                Enum.Dispose();
                return null;
            }

            Enum.Dispose();
            return device;
        }

        private static void WaitUnitReady(Device oDevice)
        {
            int eError = oDevice.UnitReady;
            while (eError != (int)DeviceError.Success)
            {
                Console.WriteLine("Unit not ready: {0}",oDevice.LastError.ToString("x"));
                System.Threading.Thread.Sleep(1000);
                eError = oDevice.UnitReady;
            }

            Console.WriteLine("Unit is ready.");
        }

        /////////////
        // Erase
        private static void Erase(Device  oDevice)
        {
            MediaProfile mp = oDevice.MediaProfile;
            switch(mp)
            {
                // DVD+RW (needs to be formatted before the disc can be used)
                case MediaProfile.DvdPlusRw:
                {
                    Console.WriteLine("Formatting...");

                    BgFormatStatus fmt = oDevice.BgFormatStatus;
                    switch(fmt)
                    {
                        case BgFormatStatus.NotFormatted:
                            oDevice.Format(FormatType.DvdPlusRwFull);
                            break;
                        case BgFormatStatus.Partial:
                            oDevice.Format(FormatType.DvdPlusRwRestart);
                            break;
                    }
                }
                break;

                // DVD-RW, Sequential Recording (default for new DVD-RW)
                case MediaProfile.DvdMinusRwSeq:
                    Console.WriteLine("Erasing...");
                    oDevice.Erase(EraseType.Minimal);
                    break;

                // DVD-RW, Restricted Overwrite (DVD-RW was formatted initially)
                case MediaProfile.DvdMinusRwRo:
                    Console.WriteLine("Formatting...\n");
                    oDevice.Format(FormatType.DvdMinusRwQuick);
                    break;

                case MediaProfile.CdRw:
                    Console.WriteLine("Erasing...");
                    oDevice.Erase(EraseType.Minimal);
                    break;
            }

            // Must be DVD-R, DVD+R or CD-R
        }

        private static DataFile CreateFileSystemTree()
        {
            DataFile oRoot = Library.CreateDataFile();

            oRoot.IsDirectory = true;
            oRoot.FilePath = "\\";
            oRoot.LongFilename = "\\";

            for (int i = 0; i < _sourceFiles.Count; i++)
            {
                TFile file  = _sourceFiles[i];

                DataFile df = Library.CreateDataFile();

                    // it is a file
                    df.IsDirectory = false;

                    // filename long and short
                    df.LongFilename = Path.GetFileName(file.Path);
                    df.ShortFilename = "";

                    df.DataSource = DataSourceType.Disc;	      // it is already on the cd/dvd
                    df.DiscAddress = file.Extents[0].DiscAddress; // set the disc address from the first extent
                    df.FileSize = file.Extents[0].DiscSize;		  // and the size

                    if (file.Extents.Count > 1)
                    {
                        // We have multiple extents. We need to add them to DataFile.UdfFileProps.Extents
                        for (int j = 1; j < file.Extents.Count; j++)
                        {
                            UdfExtent ext = Library.CreateUdfExtent();

                                ext.Address = file.Extents[j].DiscAddress; // set the disc address
                                ext.Length = file.Extents[j].DiscSize; // and the size
                                
                                df.UdfFileProps.Extents.Add(ext);

                            ext.Dispose();
                        }
                    }

                    oRoot.Children.Add(df);

                // Must dispose the object
                df.Dispose();
            }

            return oRoot;
        }

        private static void PrintError(PrimoSoftware.Burner.BlockDevice oBlockDevice, Device  oDevice)
        {
            BlockDeviceError eError = oBlockDevice.LastError;
            int iSystemError = 0;
            
            if (BlockDeviceError.SystemError == eError)
            {
                iSystemError = oBlockDevice.LastSystemError;

                StringBuilder sMessage = new StringBuilder(1024);
                FormatMessage(0x1000, IntPtr.Zero, (int)iSystemError, 0, sMessage, 1024, IntPtr.Zero);

                Console.WriteLine("IDataDisc System Error: {0} - {1}", iSystemError, sMessage);
            }
            else if (BlockDeviceError.DeviceError == eError && null != oDevice)
            {
                int iDeviceError = oDevice.LastError;

                if ( (int)DeviceError.SystemError == iDeviceError)
                {
                    iSystemError = oDevice.LastSystemError;
                    StringBuilder sMessage = new StringBuilder(1024);
                    FormatMessage(0x1000, IntPtr.Zero, iSystemError, 0, sMessage, 1024, IntPtr.Zero);

                    Console.WriteLine("IDevice System Error: {0} - {1}", (int)iSystemError, sMessage);
                }
                else
                    Console.WriteLine("IDevice Error: {0}", oDevice.LastError);
            }
            else
                Console.WriteLine("IDataDisc Error: {0}", (int)eError);
        }


        private static bool BurnFile(PrimoSoftware.Burner.BlockDevice bd, int fileIndex, long offset, long length)
        {
            // Get the start address at which BlockDevice will start writing.
            int discAddress = bd.WriteAddress;

            FileStream file = null;
            
            try
            {
                // Open file
                file = File.OpenRead(_sourceFiles[fileIndex].Path);
                file.Seek(offset, SeekOrigin.Begin);
            }
            catch (IOException ex)
            {
                Debug.WriteLine(ex.Message);
                return false;
            }

            // Set up write progress counters
            long current = 0;
            long fileSize = (long)length;

            // Allocate read buffer
            byte[] buffer = new byte[(int)BlockSize.Dvd * BLOCKS_PER_WRITE]; 

            // Write the data
            while(current < fileSize)
            {
                int read = file.Read(buffer, 0, (int)BlockSize.Dvd * BLOCKS_PER_WRITE);
                int written = 0;
                if (read != 0)
                {
                    // Align on 2048 bytes
                    if ((read % ((int)BlockSize.Dvd)) != 0)
                        read += (int)BlockSize.Dvd - (read % (int)BlockSize.Dvd);  

                    // Write the data
                    if(!bd.Write(buffer, (int)read, ref written))
                        break;

                    if (0 == written)
                        break;
                }
                    
                // Update current position (bytes)
                current += (long)written;
            }

            // Close file
            file.Close();

            // Create new extent
            TExtent ext = new TExtent();
            ext.DiscAddress = discAddress;
            ext.DiscSize = fileSize;

            // add the new extent to the list
            _sourceFiles[fileIndex].Extents.Add(ext);

            return true;
        }

        private static bool Burn(PrimoSoftware.Burner.BlockDevice bd)
        {
            // Open block device 
            if (!bd.Open())
                return false;

            // Burn fileSize files that the user specified. 
            // The code below demonstrates how you can burn the first few bytes of a file last.
            // For demo purposes split each file in half and burn first half after the second.
            for (int i = 0; i < _sourceFiles.Count; i++)
            {
                FileInfo fi = new FileInfo(_sourceFiles[i].Path);

                // Calculate the first half size and align it to 16 * BlockSize.Dvd(2048).
                // The alignment to 16 * BlockSize.Dvd must be done for all partitions except the last one.
                long firstHalfSize = (fi.Length / 2);
                firstHalfSize = (firstHalfSize / (16 * (long) BlockSize.Dvd)) * (16 * (long) BlockSize.Dvd);

                // Calculate the second half size. Since it is the last partiiton of the file we do not have to align it to 2048 byte
                long secondHalfSize = fi.Length - firstHalfSize;

                // Burn second partion first. BurnFile will add a new extent to _sourceFiles[i].
                BurnFile(bd, i, firstHalfSize, secondHalfSize);

                // Burn first partition last. BurnFile will add a new extent to _sourceFiles[i].
                BurnFile(bd, i, 0, firstHalfSize);

                // IMPORTANT: Reverse the extents here so the extent of the second partition comes after the extent of the first partition 
                _sourceFiles[i].Extents.Reverse();
            }

            // Close block device
            if (!bd.Close())
                return false;

            // Create files system tree
            DataFile fileSystem = CreateFileSystemTree();

            // Finalize disc
            if (!bd.FinalizeDisc(fileSystem, "HPCDEDISC", true, true, false))
            {
                fileSystem.Dispose();
                return false;
            }

            fileSystem.Dispose();
            return true;
        }

        private static bool Burn(Device dev)
        {
            Debug.Assert((dev != null));

            // Set write speed
            int maxSpeedKB = dev.MaxWriteSpeedKB;
            if (dev.MediaIsDVD) 
            {
                Console.WriteLine("Setting write speed to {0}x", Math.Round((double)maxSpeedKB / Speed1xKB.DVD, 1));
                dev.WriteSpeedKB = maxSpeedKB;
            }
            else
            {
                Console.WriteLine("Setting write speed to {0}x", Math.Round((double)maxSpeedKB / Speed1xKB.CD));
                dev.WriteSpeedKB = maxSpeedKB;
            }

            //Create block device object
            PrimoSoftware.Burner.BlockDevice bd = Library.CreateBlockDevice();
            
            // Set device
            bd.Device = dev;

            // Set temporary disc ID
            bd.TempDiscID = "DISCID";

            // Burn 
            bool bRes = Burn(bd);

            // Check for errors
            if (!bRes) 
            {
                PrintError(bd, dev);
        
                bd.Dispose();
                return false;
            }

            bd.Dispose();
            
            return true;
        }

        private static void ReadDiscID(Device dev)
        {
            //Create block device object
            PrimoSoftware.Burner.BlockDevice bd = Library.CreateBlockDevice();
            
            //Set device
            bd.Device = dev;

            // Open block device 
            if (bd.Open(BlockDeviceOpenFlags.Read))
            {
                // Show information about the disc in the device
                Console.WriteLine();
                Console.WriteLine("Disc Info");
                Console.WriteLine("---------");

                Console.WriteLine("Finalized: {0}", bd.IsFinalized);
                Console.WriteLine("Temporary Disc ID: {0}", bd.TempDiscID);
                Console.WriteLine();

                // Close block device
                bd.Close();
            }

            bd.Dispose();
        }


        [STAThread]
        static int Main(string[] args)
        {
            int iParseResult = ParseCommandLine(args);
            
            if (0 != iParseResult)
                return iParseResult;

            //////////////////////////////////////////////////////////////////////////////////////////
            // 1) Create an engine object

            // Create an instance of IEngine. That is the main iterface that can be used to enumerate 
            // fileSize devices in the system
            Library.Initialize();
            Engine oEngine = Library.CreateEngine();

            ///////////////////////////////////////////////////////////////////////////////////////////
            // 2) Inititialize the engine object

            // Initialize the engine
            oEngine.Initialize();

            Library.EnableTraceLog(null, true);

            ///////////////////////////////////////////////////////////////////////////////////////////
            // 3) Select a device (CD/DVD-RW drive)
    
            // The SelectDevice function allows the user to select a device and then 
            // returns an instance of IDevice.
            Device oDevice = SelectDevice(oEngine);
            if (oDevice == null)
            {
                // Something went wrong. Shutdown the engine.
                oEngine.Shutdown();

                // Release the engine instance. That will free any allocated resources
                oEngine.Dispose();

                // We are done.
                Library.Shutdown();
                return -1;
            }

            // Close the device tray and refresh disc information
            if (oDevice.Eject(false))
            {
                // Wait for the device to become ready
                WaitUnitReady(oDevice);

                // Refresh disc information. Need to call this method when media changes
                oDevice.Refresh();
            }

            // Check if disc is present
            if (MediumReady.MediumPresent != oDevice.MediumReady)
            {
                Console.WriteLine("Please insert a blank disc in the device and try again.");
                oDevice.Dispose();

                oEngine.Shutdown();
                oEngine.Dispose();
                Library.Shutdown();
                return -1;		
            }

            // Do the work now
            switch (_burnOption)
            {
                case BurnOption.Erase:
                    Erase(oDevice);
                break;
                case BurnOption.ReadDiscID:
                    ReadDiscID(oDevice);
                break;
                default:
                {
                    // Check if disc is blank
                    if (MediaProfile.DvdPlusRw != oDevice.MediaProfile && !oDevice.MediaIsBlank)
                    {
                        Console.WriteLine("Please insert a blank disc in the device and try again.");
                        
                        oDevice.Dispose();

                        oEngine.Shutdown();
                        oEngine.Dispose();

                        Library.Shutdown();
                        return -1;		
                    }

                    Burn(oDevice);
                }
                break;
            }

            // Dismount the device volume. This forces the operating system to refresh the CD file system.
            oDevice.Dismount();

            // Release IDevice object
            oDevice.Dispose();

            // Shutdown the engine
            oEngine.Shutdown();

            // Release the engine instance
            oEngine.Dispose();

            Library.DisableTraceLog();

            Library.Shutdown();

            return 0;
        }

        private static List<TFile> _sourceFiles = new List<TFile>();
        private static BurnOption _burnOption = BurnOption.Unknown;

        [DllImport("msvcrt.dll", EntryPoint = "_getch")]
        protected static extern int getch();

        [DllImport("kernel32.dll")]
        public static extern int FormatMessage(int lFlags, IntPtr lSource, int lMessageId, int lLanguageId, StringBuilder sBuffer, int lSize, IntPtr lArguments);
    }
}

@fengjiannan2010
Copy link
Author

fengjiannan2010 commented Apr 30, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants