Blackbanded Sunfish, Enneacanthus chaetodon
A 30-60mm long fish with a very compressed, deep body, a thin pointed pectoral fin and round tail fin. Habitat: swamps, ponds and river pools.
Sunfish
An NFS client, implemented as a filing system. Habitat: RISC OS.
Contents |
Introduction |
Sunfish is a filing system (and an image filing system) for RISC OS that allows you to access NFS servers on other platforms. The frontend allows you to browse the local network for any suitable servers, and then provides a means of mounting any directory that the server exports.
Sunfish provides a range of options for connecting to the server, and controlling things like mapping of filetypes and translation of filenames. However in most cases the default settings should work satisfactorily, and so you should not need to change anything unless you have a specific need.
License |
Sunfish is copyright © 2003-2010 Alex Waugh
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
Sunfish also includes some code (memcpy.s) taken from UnixLib.
Copyright (c) 1995-2005 UnixLib Developers
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Requirements |
Sunfish requires a working network connection and an NFS server to connect to. The server should be running version 2 or version 3 of the NFS protocol, version 1 or 3 of the Mount protocol, version 2 of the portmapper protocol, and optionally version 2 of the pcnfsd protocol. Both UDP and TCP connections are supported.
Sunfish requires the Iconv module to be installed. This can be downloaded from here.
Sunfish should run on any version of RISC OS from 3.11 onwards. It has been reported to work on RISC OS 3.7, RISC OS 4 (including under Virtual RPC), Select, and RISC OS 5. It is 26/32bit neutral.
Sunfish has been reported to work with the following servers:
| Server | Notes |
| Moonfish on RISC OS | No known issues |
| Linux 2.4 kernel server | No known issues |
| Linux 2.6 kernel server | No known issues |
| Solaris 8 server | No known issues |
| TrueGrid NFS server on Windows XP | Question marks in filenames (and possibly other characters) can cause "No such device" errors. Setting the Translate characters that are illegal on Windows option avoids this. |
| Netware 6.5SP1a. | No known issues |
| Allegro NFSd on Windows | Version 1.1.4 has a bug in that can cause file not found errors when copying files onto the server. Allegedly fixed in later versions. A similar problem still exists when renaming files. |
| Microsoft Services for Unix 3.5 on Windows XP | Some characters in filenames can cause "Permission denied" errors. Setting the Translate characters that are illegal on Windows option avoids this. |
You may also have to ensure that the connection is not blocked by a firewall. The default configuration on some RedHat systems is known to block NFS traffic.
The rest of this document assumes your server is correctly setup. You may wish to refer to Allan Curtis' notes on setting up a Linux NFS server, or the Drobe article on setting up the TrueGrid server for use with Sunfish (although be aware that this refers to an older version of the Sunfish frontend).
Any updates will be available from http://www.alexwaugh.com/
Basic operation |
Double click on !Sunfish to load the frontend. An icon will appear at the left hard side of the iconbar. Click on the icon to open the server browser window. This is similar in behaviour to a standard filer window, and it shows all the servers that can be found on the local network. If the server is on a different subnet then it won't be found, so click menu over the window and follow the "Search for host" entry. Enter the hostname or IP address of the server. If the server is found, it will appear in the server browser window.
Doubleclicking on the server's icon in the server browser window will open the export browser window, which shows all the directories that are exported by the server. Doubleclick on an export to mount it, which will open the identification window.
The NFS protocol requires that all accesses specify a user identification number (UID) and group identification number (GID) to identify who is performing the access, and which files they are allowed to access.
On Unix like servers, you can normally find the UID and GID of a particular user by looking in the /etc/passwd file. On some servers which don't have a concept of multiple users (for example the Moonfish server running on RISC OS) then the value of the UID and GID does not matter, and the fields can be left blank in order to use the default UID and GID.
If the server is running pcnfsd (which is fairly unlikely) then you will be asked for a username and password instead which pcnfsd will translate to a UID and GID.
Clicking Set will use the specified UID and GID for this session, and next time you try to mount the export you will be asked again. Clicking Save will save the values, and next time you mount the export the saved values will be used automatically. Once either Set or Save have been clicked, the iconbar icon will change to identify the export, and a filer window will open showing the contents of the exported directory. You can then use this filer window as you would a local disc.
Using the frontend |
When first loaded, and there are no directories mounted, the iconbar icon will be the Sunfish icon with no text underneath. Clicking the Select mouse button on the icon will open the server browser window. Once a directory has been mounted, the icon will change to the Sunfish directory icon, with text underneath showing the name of the mount. If more than one directory is mounted there there will be a separate icon for each directory. Clicking Select will open a filer window showing the contents of the mounted directory. All other behaviour is the same between the mounted and unmounted icons, although some menu entries may be unavailable in the latter case.
Clicking the Adjust mouse button on an iconbar icon will open the server browser window. The server browser window may also be opened by selecting the Browse... entry from the iconbar icon.
The Save mounts entry on the iconbar menu will save a list of the directories that are currently mounted. When you next load Sunfish, this list will be read and any directory in it will automatically be mounted.
The Dismount entry on the iconbar menu will dismount the directory so that its contents can no longer be accessed, and its icon will be removed from the iconbar.
The Free... entry on the iconbar menu will show the amount of free and used space for the disc containing that directory. The meaning of the numbers returned may vary on different types of server, but typically they will be the free and used space of the entire disc or partition, even if the exported directory only takes up a small portion of the disc.
The Quit entry on the iconbar menu will quit the frontend. However the module will remain active so all the mounted directories and image files currently in use will continue to work.
The server browser shows all the servers that have been found. A server will be found if it has a portmapper service running, so it is possible for servers to be listed even if they don't have an NFS server running (in this case you would get an error when you doubleclick on the server's icon).
Servers are found by sending out a broadcast request on the local subnet, therefore any servers that are not on the local subnet will not be found. If the server you wish to use is not on the local subnet then you must click menu over the window, follow the Search for host entry and enter the hostname or IP address of the server. If the server responds then an icon for it will be added to the server browser window.
Doubleclicking on a server icon will open the export browser window which shows the directories that the server is exporting. Sunfish supports the NFS 2 and NFS 3 protocols, both over TCP or UDP transports. At this point it will pick the best protocol and transport that the server supports (it prefers NFS 3 over NFS 2, and TCP over UDP). If you want to choose manually, then click menu over the server icon, and follow the Browse entry which allows you to select from any of the options that the server supports.
The export browser shows all the directories that the server is exporting. Doubleclicking on a directory will mount it using the default options. To change the options, click menu over the directory, follow the Edit entry and select one of the four option categories to change.
You can also drag a directory from the export browser window to a normal filer window to create an image filing system mount file.
By default, mounts will be named based on the hostname or IP address and export name. For example Sunfish::mint/home/alex.$. The name mount window allows you to specify a different name to use as the discname for the mount, which can make the path shorter or more meaningful. For example specifying the discname as alex would result in a pathname of Sunfish::alex.$.
Note that the choices only take effect when you next mount the export.
Filenames on different systems have different restrictions on what characters can be allowed in a filename, and how characters are encoded, therefore Sunfish can perform a number of transformations of filenames.
When converting filenames to RISC OS, and characters that are illegal in RISC OS filenames are changed into escape sequences of ?xx where xx is the hexadecimal value of the character. Spaces are mapped onto hard spaces, dots map to slashes and vice versa, and other characters are unchanged. The opposite transformation is performed when converting in the other direction.
If the Translate characters that are illegal on Windows option is set then when converting filenames from RISC OS, the ? < > characters will be translated to # $ ^ respectively. This matches the method that LanMan98 uses.
The character encoding of the filename is not specified in the NFS 2 and NFS 3 specifications, therefore by default Sunfish does not change the encoding of the filename. However if the server is using a different encoding from the client then the Encoding option can be used to specificy what encoding the server uses.
The local encoding is determined from the *alphabet setting, and if the two encodings are different then filenames are converted between the two. Common encodings are ISO-8859-1 (for Latin1) and UTF-8.
On Unix systems, filenames that begin with a dot are conventionally treated as hidden files. If the Always show hidden files option is selected then these files will appear in filer windows. If the Never show hidden files option is set then these files will not appear in filer windows and directory listings. Note that while you can read and write hidden files explicitly if you know their name, they won't show up in directory lists, and if you copy a directory any hidden files inside will not get copied.
If the Show hidden files except in root of mount option is selected then all files beginning with a . in the root directory of the mount will be hidden, but those in other directories will still be shown.
Unix like systems support symbolic or soft links. If the Follow symbolic links option is set the Sunfish will follow the link, and reading or writing to the link will actually read or write the file pointed to by the link. If the link points to a file that doesn't exist, or if the option is not set, then the link will be shown as a file that you cannot read or write. Deleting a link will only delete the link itself and not the target of the link.
Note that symlinks can only be followed if they point to a file within the mount directory or its children.
If the Case sensitive option is set, filenames will be treated as case sensitive, and if the filename doesn't match exactly then you will get a file not found error. If the option is not set, then if an exact match for the filename isn't found then a case insensitive match will be tried as well.
If case sensitivity is off, and you have two (or more) files in the same directory that differ only in case then trying to access one of them may result in the wrong file being altered. Therefore is it recommended not to have filenames that differ only in case.
Other systems do not have a concept of filetypes in the same way as RISC OS does, and they often use file extensions to determine type type of the file. In common with other RISC OS network filesystems, Sunfish uses a combination of MimeMap and ,xyz extensions (where the filetype is represented as a hexadecimal number following a comma).
If the Always add ,xyz extensions option is set then an ,xyz extension will always be added to a file when it is saved. Files with an ,xyz extension will get the specified filetype, those without will be looked up in MimeMap to find a suitable filetype, and if that fails then they will get the default filetype.
If the Never add ,xyz extensions option is set then an ,xyz extension will never be added to a file when it is saved, and all files will have the default filetype.
If the Add ,xyz extensions only when needed option is set then when saving a file, MimeMap will be looked up. If the filename has a dot extension which matches the filetype, then Sunfish will not add any ,xyz extension. If the filename doesn't have a dot extension, and the filetype matches the default filetype then no ,xyz will be added. Otherwise a ,xyz extension will be added. This is generally the best option to use, as it ensures cross platform files that have dot extensions get the correct filetype on RISC OS, whilst avoiding the need for a ,xyz extension.
The default filetype is normally text (&FFF) and it is not recommended to change this.
If the Set filetype to UnixEx for executable files option is set then if the file has Unix executable permission (for any of user, group or others), then it will be given a filetype of UnixEx (&FE6) overriding its normal filetype. If you create a new file with the UnixEx filetype or change an existing file's filetype to UnixEx, then the user execute bit will be set, and the group and others execute bits will be set if the corresponding read bit is set.
When reading or writing large files, the file is broken up into smaller chunks to transfer across the network. The size of the chunks has an effect on performance, and in general the bigger the better. However the some network cards (In particular, the Castle 100BaseT podule) cannot handle packets larger than 4096 bytes when using UDP.
The size is set in the Max data buffer size field. The maximum value is 8192 bytes for NFS2 or UDP connections, or 32768 for NFS3 with TCP, and it is recommeded that you set it to this if your network card can handle it (You will probably get timeout errors when reading a large file if it is too big).
Normally, only one chunk of the file is sent at a time, the next chunk is sent only when the first has been recieved. However if the Pipeline requests to increase speed option is set then the second chunk will be sent before the first has been recieved, which will often give an increase in speed on faster machines. Unfortunatly in some situations, particularly on slower machines, this option can decrease preformance significantly. Therefore it is worth trying this option, but turn it off again if it does not improve the speed.
If the server does not respond to a request, then Sunfish will periodically resend the request until it either recieves a response, or until a time limit has been reached when it will return an error (Connection timed out). The Timeout field can be used to alter the delay before the connection times out.
If the Log debug information to syslog option is set then all operations will be reported to syslog. This should only be enabled for debugging, as it significantly slows things down. Note that if one connection specifies logging, then activity on all connections will be logged while that connection is mounted.
The UID and GID are normally set from the identification window when the export is first mounted. However if they subsequently need to be changed then they can from this window. Up to 16 GIDs can be entered if needed, separated by spaces.
RISC OS file access attributes are based on public or private access for reads or writes. NFS uses the Unix convention of having read, write and executable permissions for the user, the group, and others. Sunfish must therefore convert between the two representations. The RISC OS private attributes map to Unix user permissions, and the public attributes map to the group and others permissions (if a public attribute is set then both the corresponding group and others permission bit will be set). RISC OS does not normally enforce the access permissions, so it is possible for files to not have private read or write permission, yet for programs to access them as normal. However if such a file is transferred to Unix, then the lack of permissions suddenly become a problem, so Sunfish provides the unumask option which is specified in octal like Unix file permissions traditionally are, and is logically ORed with the Unix mode bits to allow particular bits to be forced to be set. Finally, the mode bits are then logically ANDed with the inverse of the umask option (also in octal) which allows you to force specific bits clear (in the same way as the traditional umask on Unix).
The combination of the umask and unumask allow you to control exactly which bits of the unix permissions are set, cleared, or depend on the RISC OS access attributes. The default values are 0600 for the unumask and 022 for the umask, which ensures that all files created have user read and write permissions, will have group and others read permission if the public read attribute is set, and will never have group or others write permission.
Use as an image filing system |
In addition to providing a normal filing system, Sunfish also provides an image filing system. This allows you to save a mount file anywhere on disc, and access it as if it were just another directory on the disc. It also allows access to the mounts without having to load the frontend.
Mount files can be created from the frontend by opening the export browser window, setting any option you want for the mount, then dragging the export icon from the export browser window to a filer window. Alternately, a mount file can be manually created by using your favorite text editor to create a file with the appropriate options as described below. Save this, then set its filetype to Sunfish (&1b6). Then double click on the mount file you have just created and the server will be mounted and you can navigate the filesystem as normal.
Due to the way image filing systems work, you may not be able to edit the mount file once it has been mounted and while Sunfish is still running. There are two solutions - either set the filetype of the mount file to Text, edit it, then set it back to Sunfish, or *RMKill Sunfish before editing the file.
This is a very simple mount file, but is sufficient to get a working mount. The fields allowed are described below, and most of them correspond directly with the options present on the frontend. However a few of the more obscure options are not available in the frontend.
Protocol: NFS2 Server: mint.cp15.org Export: /home/ajw498 UID: 1001 GID: 50
A more complex example.
Protocol: NFS3 Server: mint.cp15.org Transport: tcp Export: /tmp UID: 1002 GID: 51 GIDs: 52 53 105 umask: 066 MachineName: caramel ShowHidden: 0 Timeout: 5 DefaultFiletype: FFF AddExt: 2 MaxDataBuffer: 32768 Pipelining: 1
Image filing system options |
Any line beginning with a # is treated as a comment.
<Inet$Hostname> which should be sufficient in most cases.
Generally, it doesn't matter what port is used, but some servers will only allow connections from ports less than 1024 unless the insecure option is used in /etc/exports on the server.
Note that while you can read and write hidden files explicitly if you know their name, they won't show up in directory lists, and if you copy a directory any hidden files inside will not get copied.
If case sensitivity is off, and you have two (or more) files in the same directory that differ only in case then trying to access one of them may result in the wrong file being altered.
? < > characters will be translated to # $ ^ respectively. If not specified, then it defaults to 0.
*alphabet setting, and if the two encodings are different then filenames are converted between the two. Common encodings are ISO-8859-1 (for Latin1) and UTF-8. If no encoding is specified, then filenames are used without any conversion.
Improving Performance |
There are four options that are likely to have an effect on performance: the protocol, the transport, the MaxDataBuffer option and the Pipelining option.
NFS3 should be faster than NFS2, because it is more efficient when listing directory contents, and allows larger data buffers.
Using TCP as the transport may give better results than UDP in most circumstances.
Normally, the bigger the data buffer the better, however some network cards (notably the Castle 100bT podule) cannot cope with large packet sizes when using UDP. I suggest you try setting it to the maximum, and if you start getting timeout errors when transferring files then reduce it.
The pipelining option (which is disabled by default) allows file read and write requests to be pipelined which can give around a 20% increase in speed. However, with some servers or network setups it could make things slower. If, for example, your server is running on a 100bT segment while the client is the other side of a switch on a 10bT segment then it is likely to make things worse on a UDP connection, but if the server and client are of comparable speeds on the same segment then it may make things faster.
The server can also have a influence on the speed.
I have found the fastest settings on an Iyonix with a 1000bT connection to be NFS3 over TCP, with a data buffer of 32768 and pipelining turned on.
The following table shows the maximum speeds I have obtained with various settings, and a comparison with LanMan98 and LanManFS. If you are seeing significantly slower results then it may be worth experimenting with the above options. All the tests were copying a 125MB file using the filer with the faster option set.
| Iyonix 1000bT | Iyonix 100bT | SA RPC, Simtec 100bT | ||||
|---|---|---|---|---|---|---|
| Read | Write | Read | Write | Read | Write | |
| Sunfish NFS2 UDP | 9.2MB/s | 10.3MB/s | 7.3MB/s | 8.8MB/s | 1.2MB/s | 1.2MB/s |
| Sunfish NFS3 TCP | 9.5MB/s | 11.2MB/s | 5.9MB/s | 7.3MB/s | 1.2MB/s | 1.1MB/s |
| LanMan98 | 8.8MB/s | 2.0MB/s | 7.3MB/s | 4.3MB/s | 1.3MB/s | 0.78MB/s |
| LanManFS | 2.9MB/s | 5.7MB/s | 2.8MB/s | 5.9MB/s | ||
Reporting Bugs |
If you discover a bug, please report it to me, or better still fix it yourself and send me a patch. When reporting a bug, first load SysLog, then enable the logging option for the mount before doing the operation that triggers the bug. A log file should be generated in !SysLog.Logs.Sunfish. If you have access to a machine that can run a packet sniffer such as ethereal or wireshark then it would be very useful to use that to save a dump of the network traffic along with the syslog file. Also, please provide a description of the problem and exactly what operation you were doing to trigger the bug, and details of what OS and NFS version the server is running would be helpful, and the version of the Sunfish module and RISC OS version.
Known Issues |
The leafname returned from an OS_File 0 call will be in Unix format and may still have a ,xyz extension. This is only used for printing *opt 1 info so is not crucial. The PRMs state that *opt 1 may not work correctly on RISC OS 3 onwards anyway.
Wildcards in filenames are not supported. This would be a lot of hassle to implement for little benefit, and in any case I believe that it really should be fileswitch's job to sort out wildcards, not each and every filing system.
Devices and other non regular files are not supported, as they don't map to RISC OS functionality.
If ShowHidden == 0 and you try to delete a directory that contains hidden files then it will fail with a directory not empty error.
If an NFS3 server returns a jukebox error, Sunfish should retry the operation a short while later. It doesn't, and reports the error to the user. If you manually retry then things should work.
Files greater than 2GB in size are reported as having a size of exactly 2GB. The data in the first 2GB of the file can be read and written to normally, but the rest of the file cannot be accessed. This is due to RISC OS not supporting files larger than 2GB.
The server and export browser windows sometimes show icons left aligned rather than centred when running on RISC OS Select.
Recompiling |
Full source code is provided. To recompile the module, you will need at least either GCC 3.4.4 release 3 or Norcroft v5.53, cmhg or CMunge, TCPIPLibs, Perl, and GNU Make. Earlier 26bit versions of Norcroft will not work because the code makes use of 64bit integers and some C99 functions.
To recompile the frontend, you will additionally need g++ 3.4.4 release 3 or later, and the RTK library. You will have to get the RTK source from svn, and compile it yourself targetting the shared C library rather than UnixLib.
The makefile in the src directory has several targets:
make - build module with Norcroft, and frontend with g++, without optimisation.
make release - build module with Norcroft, and frontend with g++, with full optimisation.
make gcc - build with GCC and g++ without optimisation.
make gcc release - build with GCC and g++ with full optimisation.