School of Earth, Energy & Environmental Sciences

Using the web to transfer files



Last revision July 22, 2004

You can use the web server on pangea to provide access to large files with password protection. They are readable to anyone with a pangea account. Here are the basic steps:

  1. Store the files on pangea. If they are very large, do NOT store them in your home directory. Instead, put them on the /scr1 scratch disk. Remember that all files on /scr1 are automatically erased after 15 days. See this page for more information on pangea disks and their use policies.
  2. Make a directory with your name in /scr1 (all accounts have permission to put directories and files in /scr1), and store your files there. For example, assume a big data file named testfile.dat. To make a directory in /scr1 use the command:

    mkdir /scr1/username

  3. Transfer testfile.dat file to that directory. If you are transferring via ftp from another computer, remember that you should transfer all programs, data or image files in "binary" or "raw data" mode.
  4. Make a symbolic link for each file from its real location to a directory that is served by the web server. If this is a research group file, you can link into the research group web site. Or, you can link into your personal web directory in your home directory. For example, to your file into your personal web directory on pangea, which is the /WWW subdirectory of your home directory, and provide password protection for this file, so that only people with the password can see it on the web, you need to put it into a subdirectory of its own in your Web area. For the example file testfile.dat, this is done by making a subdirectory inside your personal /WWW subdirectory (which already exists for all pangea accounts). Use these commands:

    cd
    mkdir WWW/testdir
    ln -s /scr1/farrell/testfile.dat WWW/testdir/testfile.dat

  5. Set up password protection for this subdirectory. You have to create a .htaccess file in this subdirectory with appropriate commands to tell the web server that this subdirectory is password protected. Then you create a password file in this subdirectory with the encrypted password that allows access. The procedure is completely described on this web page.
  6. OR, you can rely upon "security by anonymity". Because your new web page is not linked to any other pages, it is not visible to any search engines. And presumably it won't be there very long. So you may prefer to skip the password protection altogether and just figure that no one is going to stumble across this directory in the time you have it there. Only the people you explicitly tell about the directory are likely to see it.
  7. Finally, if you give your correspondent the URL to the symbolic link to your file, he can try to download it with his web browser. HOWEVER, depending upon the settings of his browser and the filename extension you use, his browser may try to display the file rather than download it. To guarantee that he can force his browser to download it to disk, use this method.
    • First, create a very small HTML file in the same directory that contains a link to the symbolic link of the data file. For more information on creating HTML files, see this web page.

      Here is an example of a testfile.html that you might create in you WWW/testdir directory:

      <html>
      <head></head>
      <body>
      <a href="testfile.dat">Get testfile.dat</a>
      </body>
      </html>

      Next, tell your correspondent the URL for this HTML file, which is

      http://pangea.stanford.edu/~username/testdir/testfile.html

      If you are not familiar with the correspondence between your actual personal web directory on pangea and this URL, see the page on personal web sharing on pangea.

  8. Finally, tell your correspondent to open that URL for the html file in his browser, and then to use this procedure to force his browser to download the data file shown in the link, rather than try to display it on screen.
    1. For most modern browsers, if you click on the link with the RIGHT mouse button on a PC, or the combination of the CONTROL key and the LEFT (only) mouse button on a Macintosh, the browser will pop up a menu of actions that you can take on this link. One of those actions will be save to disk or download to disk. Select that action, and the file referenced by the link -- your data file -- will be downloaded to the correspondent's computer's disk, NOT displayed on the screen.
    2. Test the whole procedure by trying to download from pangea to your computer before sending the information off to your correspondent.
  1. When your correspondent has the files, delete the subdirectory and files you made in your web directory and on the /scr1 disk.

 

Comments or Questions?