The proof of concept implementation of mapping the original UTF16LE-encoded path names to hashed versions in the ISO-Latin-1 encoded space did not scale well and would invariably leak memory because there is no safe way to determine whether such a mapping was no longer needed. There appears to be no way to make this work robustly.
So, sadly, I removed Tygre's implementation.
We are back to square one with mapping UTF16LE-encoded path names, and also with mapping file/directory names longer than 107 characters (Windows names can be up to 127 characters long).
Added more diagnostic output and debug functionality, e.g. regarding how many directory caches are currently being used.
Long name directory scanning is a bit more paranoid when retrying a scan operation which might have been triggered by the server connection getting dropped.
Added the new READONLY option (why stop at 47 options when you can have 48?) which makes the file system read-only by default and does not require you to use the "Lock" shell command to achieve the same effect. Note that the shell "Lock" command cannot be used to turn the read-only mode off again.
Added support for the smb:// URI scheme, which means that instead of using "smbfs username=barney password=secret domain=workgroup //nas:445/share" you can now use "smbfs smb://workgroup;barney:secret@nas:445/share" instead. It is a bit shorter than the alternative and has an advantage in allowing you to reuse smb:// URLs which you know work well on other systems.
If the Examine/ExNext/ExAll functions need to restart directory scanning, the cache maintained by the smbfs directory reader is now always cleared.
Removed the directory cache expiration time feature, along with the just-introduced CACHEEXPIRES option. The context in which the expiration time was used did not justify it as a feature. It looked more like a bug...
Rewrote the smba_readdir() function to perform its only task in a much simpler manner. It now either fills the directory cache or retrieves the cache entries to be delivered to the caller, resuming the retrieval at a predefined position if required.
smb_proc_readdir_short() now updates the scan resume information for every directory entry it can process and actually uses the scan resume information when asked to resume scanning.
If the directory cache becomes stale before the contents have been processed, scanning will have to be restarted. smbfs now tells the server that the scan operation has been aborted before it is started again.
Directory scanning in combination with deletion operations can collapse if the directory cache becomes stale before all the relevant entries have been processed. You can now tune how long the cache will remain active, which previously was a period of 5 seconds. The current default is 10 seconds, which definitely helps. Use the new CACHEEXPIRES option to change how long the cache is considered good enough.
The ERROROUTPUT option as released in version 2.17 did not work at all. I wonder how I managed to test it successfully :-/
Rewrote the smb_proc_readdir_short() and smb_proc_readdir_long() functions so that they tell the server exactly how many directory entries they are able to receive. Previously, the server could deliver more, with interesting consequences.
smb_proc_readdir_long() now verifies that the server's response parameter and data are in order, and it also makes sure not to read more from the response data buffer than there is available when decoding directory entries.
smb_proc_readdir_long() now updates the scan resume information for every directory entry it can process.
smb_proc_readdir_long() now actually uses the scan resume information when asked to resume scanning. This means, for example, if the directory cache only has room for 60 entries, then smb_proc_readdir_long() will read up to 60 entries, get called again, and resume filling the cache with the next entry provided by the server. This never worked anywhere near correctly before.
Simplified the cache table "aging", removing duplicate code.
Verified that the CACHETABLE settings do have a positive effect on the problematic "delete directory-name all" case which almost always failed to delete the entire directory tree.
Simplified the directory cache size change functionality.
Added the CACHETABLES option, which tells smbfs how many individual directory caches it should be using (default: 1). Each directory cache can be used by one directory at a time.
If you are deleting a directory tree, including subdirectories, etc. then smbfs will be limited in keeping track of what it deleted, and what directory is next up for deletion.
With only a single directory cache, it will be limited to deleting a subdirectory's contents, but forget about its parent directory, for example, and thereby end up skipping entries which should have been deleted.
This can be annoying because you will have to repeat the delete operation several times over until everything has been deleted. Increasing the number of cache tables can help, at the expense of extra memory.
Increasing the number of cache tables can also improve performance if several programs are examining the contents of directories at the same time.
Documented the directory cache a bit better, including the reason why the cache may have been invalidated while the directory contents were being read.
The directory read operation now verifies that it does not run out of cache entries while it is processing new directory records it received from the server.
The directory read operation now both detects whether or not the last entry has been read, this information is now also provided to the function which retrieved the directory entries. This change is in support of improved directory entry caching.
The directory cache now keeps better track of what's actually in it. If a directory entry could not be extracted and decoded, the number of directory entries read would not match the number of entries available from the cache, for example.
A flag now tracks if the directory cache is valid or not. Previously, checking for a valid cache involved looking at a NULL pointer.
If allocating memory for the directory cache failed partly, then there may have been invalid string pointers in the cache table, leading to a crash when trying to release the cache. Fixed.
Adding another entry to the cache, invalidating the cache or resetting the cache now all use dedicated functions instead of directly manipulating the assorted data structures.
Found a couple more cases in which debug output did not use the correct parameter information. This happened, for example, in the SMB write operation (which did not correctly report the 64 bit seek offset) and the ACTION_SET_COMMENT implementation which did not print the comment string correctly.
Added a proper string length limit check for the text buffers used by the DateToStr() function.
Enabled profiling support for SAS/C to learn more about the most frequently-used functions and their respective running times. Added inline qualifiers to these functions, or copied the respective function to where it was called, allowing it to be inlined.
The main file system processing loop no longer calls WaitSelect() every time it needs to wait for a new packet to arrive, or for a NetBIOS "keep alive" message to come along. It now defaults to calling Wait(), which has a much lower overhead than WaitSelect(). However, periodic calls to WaitSelect() and the associated NetBIOS "keep alive" handling are scheduled regularly every 10 seconds. That way the "keep alive" messages are dealt with without spending too much time with WaitSelect().
The ACTION_SETDATE operation again changes both the file/directory creation date/time and the last modification date/time. Because some SMB servers will not return a valid modification date/time, the creation date/time will be substituted when directory entry information is converted into the appropriate AmigaDOS data structures. This is problematic if only the modification time/dated is updated by the ACTION_SETDATE operation because the change may appear not to "stick".
The SMB function which would be used to update both the modification and the creation time stamps did not take care to preserve the creation time stamp. This might have contributed to the ACTION_SETDATE operation not having a noticeable effect.
The debug code no longer uses FORMAT_DEF for converting AmigaDOS time stamp information into readable text, but uses FORMAT_DOS instead. This is an attempt to track down a hard to detect bug in the debug code which might be related to locale.library date conversion. Because the buffer size for the date and time information is limited, the locale settings might just produce too much text to fit into the buffers. But that's only a theory so far...
The functions which the debug code uses to prepare 64 bit integers and strings for display are now a bit more paranoid in watching the lengths of the resulting strings, reporting possible buffer overflows.
Went over all the debug output text format strings with a fine-toothed comb and actually found a few instances of the formatting parameters not being entirely correct.
The assert.c code makes a bit more of an effort to handle NULL strings for display.
The non-debug builds of smbfs now complain again if any of the debug parameters are being used.
The smb_receive_raw() function in "sock.c" again copies the NetBIOS header into the receive buffer separately, rolling back an earlier change which did not seem to be sound, and also served to make complicated code even more complicated :-/
Added the SCATTERGATHER tuning option which defaults to "no". Instead of breaking down write operations into two separate send() calls (one short, one very large), setting SCATTERGATHER=yes can call sendmsg() instead which allows the entire operation to be completed in one single step. This approach was suggested by Patrik Axelsson. Currently, I am uncertain if all AmiTCP V3/V4 TCP/IP stacks implement sendmsg() for TCP sockets in the same consistent manner, which is why this feature is not enabled by default.
Added the WRITETHRESHOLD parameter which can be used to disable the separate transmission of the SMB message header and its payload if the combined sizes of both are smaller than or equal to the threshold value. This is intended to improve write performance for small amounts of data which would otherwise linger in the transmission queue.
Added the READTHRESHOLD counterpart to WRITETHRESHOLD which affects the read operations.
The smb_receive_raw() function in "sock.c" no longer copies the NetBIOS header into the receive buffer separately, but makes sure that the initial receive operation takes care of it.
The allocate_path_name() function in "smb_abstraction.c" failed to put the path separator character where it should be, resulting in the path name and the file/directory name to get lumped together. This affected creation of files and directories both.
The default values for READTHRESHOLD and WRITETHRESHOLD now have the effect of transmitting the SMB header and the payload separately, just like it was introduces in version 2.1.
Use something like READTHRESHOLD=1500 and WRITETHRESHOLD=1500 to send packets smaller than or equal to 1500 bytes as a combined lump of data, and data larger than 1500 bytes as separate SMB header and payload.
When processing the share name the limitations on the individual components (server name, share, port number/IP service name) are no longer verified using hard-coded lengths, but use the respective buffer limits instead. The port number/IP service name length check is new.
Added more debug log output to the code which now performs the file and lock name/address lookups instead of walking throug the entire file/lock lists.
Added const qualifiers to the parameters used by the functions in "crypt.c". This in turn revealed that the smb_encrypt() function did not in fact modify the password string passed to it, which allowed the smb_proc_reconnect() function in "proc.c" to be simplified.
Augmented the file handle, file lock and SMB file access operations with splay trees to improve performance for validation and lookup. Instead of having to walk the entire list of file, lock or SMB file entries, comparing the list entries by name or address, the entire process should have to check 1-5 entries only.
This splay tree functionality is a compile time option which has to be enabled in the "splay.h" header file.
If there is a port number given as part of the share name which is out of range, smbfs now reports this as a specific error rather than just stating that the service is not valid.
Replaced duplicate code in "smb_abstraction.c" which built path names from a parent directory name and a file/directory name with a single common solution.
Revised the build makefiles to create the standard smbfs variant rather than the one with full debugging support enabled. The AmigaOS4 variant specifically avoids using cross-compilation options which would have to be enabled first, if needed.
Added the ADDVOLUME option, which defaults to "yes". If no volume name is
provided, smbfs will use the service name as the template, e.g. if you
connect to //server-name/pictures then the default volume name will
be "pictures". The idea is that if the file server allows this, you
should not need to start the smbfs program with any parameter other
than the service/share name and it should work out of the box.
Preparations for showing error messages when running as a Workbench
program would corrupt memory once the message was ready for display,
but this only happened in the AmigaOS4 build. As far as I can tell
this never worked correctly on AmigaOS4. Thanks go to Matthew Kille
who reported the problem!
Simplified the icon parameter processing used when running smbfs
as a Workbench program. The reworked code is now basically limited
to processing and setting up the same configuration data which the
shell command use employs. Both paths share the same configuration
data checking and handling.
Turns out that the ACTION_CURRENT_VOLUME function does need to return
the volume node, not the device node. Note to self: a 'struct DeviceList'
refers to a volume (DLT_VOLUME), whereas a 'struct DeviceNode' refers
to a device (DLT_DEVICE). See? This could have been easily been avoided :-/
Added more code documentation, especially for the various constants and
data structures defined and used in "main.c".
Reading from the file name translation file now performs better error
detection and reporting. If read errors should crop up, the error message
will now say so. If the file is shorter than required the error message
will reflect this.
Reworked the smb_proc_reconnect() function so that it does not end up
truncating the password before it is encrypted. This means that up to
63 characters of the password can be used to produce the encrypted
form, resolving problems with longer server passwords not quite
matching what you entered. However, since this long encrypted password
is still accompanied by the legacy encrypted form which truncates the
password to 14 characters (which are converted to all-upper-case
letters) there is no actual gain in terms of security...
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Please keep in mind that this is still a development version and might surprise you (not necessarily in a good way).
Do not let me discourage you to build and test this version, although there will be some risks involved such as data corruption or loss of data.
Olaf Barthel
Tygre
obarthel