GNU/Linux

Update or Install Applications on Debian/Ubuntu Without an Internet Connection

27

Update: There is a simpler way for the Ubuntu 6.10(Edgy Eft) users:
Simple Way to Update Ubuntu Edgy With Slow/No Internet Connection

If you use a Debian based system(Ubuntu etc) with a slow/no Internet connection, you will be limited to installing only those applications that come with the installation CD/DVD set. The issue is more serious with Ubuntu like operating systems where there is only one installation CD/DVD. But there is a way in which the required files can be downloaded on a machine with fast Internet connection and then copied over to your local machine(using a media like CD etc) and make them available to the Advanced Package Manager(apt). Below is the complete procedure to do the same.

Note to Ubuntu users: You need to add “sudo” before all the commands specified below.

  1. On your local machine, run the following command to generate a list of files needed for the application(s) you wish to install:
    sh# apt-get –qq ––print-uris install {packagelist} | awk -F\‘ ‘{ print $2}‘ > get.lst
  2. The URIs from where to download all the files needed to install the packages listed in {packagelist} will be added to the get.lst file. Copy this file over to the machine with fast Internet connection and run the following commnad there:
    sh# wget -c -i get.lst

    This will download all the .deb files listed in get.lst file in the current directory. Copy all these .deb files to your local machine in a directory, say /root/sharedebs.

  3. Move to /root directory and run the following command:
    sh# dpkg-scanpackages sharedebs /dev/null | gzip > sharedebs/Packages.gz
  4. Now add the following line to your /etc/apt/sources.list file:

    deb file:/root sharedebs/

    Run the command ‘apt-get update’ once to update the apt cache.

Now you can install the applications in the normal way by running the
‘apt-get install’ command or using Synaptic Package Manager like tools.

You can also use the above procedure to share the .deb files that you have downloaded on your machine to install any applications with another machine(say your friend’s machine) if you want to install the same applications on it without having to download the same .deb files again. Just copy all your .deb files from /var/cache/apt/archives/partial/ to your friend’s machine(in /root/sharedebs directory) and follow the steps from Step 3 on your friend’s machine.

Here is an example of how you can install the package lynx on your local machine using the above procedure:

  1. On local machine:
    sh# apt-get -qq ––print-uris install lynx | awk -F\‘ ‘{ print $2 }‘ > get.lst
    sh# cp get.lst /media/cdrom
    sh# umount /media/cdrom
  2. Update: If you are getting errors with the above awk command, you can try with a perl script. More details in this comment.

  3. Copy the file get.lst to the machine with fast Internet connection(using a CD etc):
    sh# mkdir /home/user/debs
    sh# cp /media/cdrom/get.lst /home/user/debs
    sh# cd /home/user/debs
    sh# wget -c -i get.lst
    sh# cp *.deb /media/cdrom
    sh# umount /media/cdrom
  4. Copy all the .deb files to the local machine(again, using CD etc) in
    /root/debshare directory:

    sh# mkdir /root/debshare
    sh# cp /media/cdrom/*.deb /root/debshare
    sh# cd /root
    sh# dpkg-scanpackages sharedebs /dev/null | gzip > sharedebs/Packages.gz
  5. Add a line in the /etc/sources.list file and install the application using apt-get:
    sh# emacs -nw /etc/apt/sources.list

    deb file:/root sharedebs/

    sh# apt-get update
    sh# apt-get install lynx

Update or Install Applications on a Gentoo Machine Without an Internet Connection

2

If you use Gentoo on a computer that doesn’t has a fast Internet connection, you can still update it or install new applications on it, by downloading files on a different system that has a fast Internet connection and then copying them to your machine using media like CD, DVD or a USB Pen Drive. The complete procedure is given below.

  1. Download the following shell script and copy it to the home directory of the local machine:
    shotsync.sh
    If the browser displays the contents of the file, use File->Save As… menu item to save the file. Or right-click on the link and select “Save Target/Link as…” from the context menu.
    We’ll need this script in Step 4.
  2. Also download the following file on the machine with fast Internet connection:
    dist-grabber.py

    Make the file executable and place it in a directory that is in the system PATH.

    sh# chmod +x distfile-grabber.py
    sh# mv distfile-grabber.py /usr/bin

    We’ll need this script file in Step 6.

  3. On the machine that has a fast Internet connection, download the latest snapshot of the portage tree from:
    Gentoo Portage Snapshots

    Let’s call this file ‘snapshot.tar.bzip’.

  4. Copy this file in the home directory of the local machine. From the home directory on the local machine, run the following command:
    sh# shotsync.sh snapshot.tar.bzip

    (we had downloaded the shotsync.sh file in Step 1)

  5. On the local machine, create a list of files to be downloaded on the other machine using the following command:
    sh# emerge -uDp –fetchonly world 2> /tmp/distfiles.txt

    (that’s two hyphens(-) for ‘fetchonly’ option)

    (replace ‘world’ with a package name if you don’t want to do a complete upgrade: sh# emerge -uDp –fetchonly lynx 2> /tmp/distfiles.txt )

  6. Copy the file generated in the above step(distfile.txt) to the machine with good Internet connection. Run the following command to download all the packages specified in the distfile.txt file:
    sh# distfile-grabber distfiles.txt

    (we had downloaded distfile-grabber.py script in Step 2)

  7. Copy all the above downloaded files into the /usr/portage/distfiles directory of the local machine. Then perform the final installation using the following command:
    sh# emerge -uD world

    (If you had used a package name in Step 5, use the same here too: sh# emerge -uD lynx)

Repeat Steps 3-7 at regular intervals to keep your Gentoo system updated. If you only want to install applications without doing a complete update of the portage tree(eg. its already updated), you can just follow Steps 5-7 by replacing ‘world’ with the name of the package.

Whenever you need to copy a file from local machine to the machine with fast Internet connection or vice versa, you can use any suitable media like CD, DVD, USB Pen Drive, Portable Hard disk etc.

More info: Gentoo Wiki
All the scripts mentioned in this post are taken from this source.

NOTE 1:
If the machine with fast Internet connection uses the Windows operating system, you can still use the dist-grabber.py script to download the packages, provided you have Python installed on that system(and is in the environment path). If you instead have Perl installed on the system, you can make use of the following perl script for the same job:
dist-grabber.pl

NOTE 2:
If you would rather use wget to download the package files, then get the required URLs, convert the URL list to wget format & then get the package files:

sh# emerge –pretend –fetchonly –update world 2> list.txt
sh# cat list.txt | sed ‘s/\shttp:/\nhttp:/gi’ | sed ‘s/\sftp:/\nftp:/gi’ > wgetlist.txt
sh# wget -i wgetlist.txt -nc

(that’s two hyphens(-) for ‘pretend’ and ‘fetchonly’ options)

Apache + PHP + MySQL on Gentoo

5

Most of the today's web applications need the combo of Apache, PHP and MySQL packages to be first installed on the system. Installing these software on a Gentoo system should be a straightforward process. Below is the complete procedure.

First add apache2 and mysql to the USE flag in the /etc/make.conf file:

sh# emacs -nw /etc/make.conf

USE="gtk gnome qt kde dvd alsa cdr hal howl imap maildir mysql apache2"

emerge the apache package:

sh# emerge apache
sh# /etc/init.d/apache2 start
sh# rc-update add apache2 default

Test the installation by accessing the url http://localhost/ from a web browser. If you get a welcome message from apache, the installation is complete. More info here.

Now emerge PHP4 and the mod_php module for apache:

sh# emerge dev-php/php mod_php

Add the configuration directive(-D PHP4) to /etc/conf.d/apache2 file and restart the apache web server:

APACHE2_OPTS="-D DEFAULT_HOST -D PHP4"

sh# /etc/init.d/apache2 restart

Create a sample php script in the directory /var/www/localhost/htdocs with the following contents and access it from the web browser to test the php installation:

[php] phpinfo();
?>[/php]

If you see a lot of information displayed about the version of PHP and other related things then PHP is installed and properly configured with the apache web server. Installation of PHP5 is also similar but be warned that still a lot of software runs only on PHP4(Eg: phpBB Bulletin Software). With a little bit of effort you can also run both PHP4 and PHP5 on the same system side by side.

Install MySQL and all the related tools:

Add the following lines to the /etc/portage/package.keywords file:

>=dev-db/mysql-administrator-1.1.5 ~x86
>=dev-db/mysql-query-browser-1.1.17 ~x86
>=dev-cpp/gtkmm-2.8 ~x86
>=dev-cpp glibmm-2.7 ~x86

sh# emerge mysql mysql-administrator mysql-query-browser
sh# emerge –config mysql-4.1.14
sh# /etc/init.d/mysql start

Set a root password for mysql server when prompted by the 'emerge –config' command. You can administer the mysql server by using either the command line tools or using the graphical tool mysql-adminstrator. Similarly, the command line client mysql can be used to create and view the database schemas or the graphical tool mysql-query-browser can be used. I recommed you give these two graphical applications a try.

Other databases, like PostgreSQL, can be used instead of MySQL whenever the applications support it, but a majority of the applications support only the MySQL database. WordPress for example doesn't support PostgreSQL.

A NFS Story on Gentoo

0

I still carry the habit of storing all the large files in my Windows partitions from my bad old Windows days. That allows me to plug any kind of device into the computer and copy the files to/from it. Gnu/Linux supports all kinds of DVD writers and Pen drives too, but my friends sometimes carry such wierd devices(and mp3 players) that they can be connected only to a computer using the Windows operating system. Accessing files in the Windows OS from a GNU/Linux partition is a time consuming and inefficient process – I do it using the excellent tool called Explore2FS, but its not an ideal tool for copying large amounts of files and directories. So it suits me well: store the files on the Windows partitions and access them easily from both Windows and GNU/Linux(by using a simple mount command).

When I discovered that my old Debian Sarge installation was not detecting the USB port for some reason and that its DVD-ROM has also died upon me, my only way to transfer some files to it was through the LAN. The files were contained in an ISO file on a Windows NTFS partition on a different machine. Below is the description of what I had to do to make the ISO file contents available to my USB-less, DVD-less Debian system.

I use Gentoo as my primary operating system on my personal desktop. The first job for me was to mount the local Windows partition(hda7) that contained the needed ISO file in the Gentoo filesystem.

sh# mkdir /mnt/win_d
sh# mount /dev/hda7 /mnt/win_d
sh# ls /mnt/win_d/*.iso

The ISO file was visible. I tried to export the /mnt/win_d directory using NFS and then tried to mount the ISO file on the Debian system, but it didn’t work out that way for me. Not in a mood to loose too much time over it, I mounted the ISO file on my Gentoo System itself and then exported this mounted directory:

sh# mkdir /mnt/myiso
sh# mount -o loop /mnt/win_d/myiso.iso /mnt/myiso
sh# ls /mnt/myiso

All the files contained in the ISO file were visible. The next job is to install the NFS server and the related tools if they are already not available, and then export the /mnt/myiso directory using it.

sh# emerge nfs-utils

Put the following line /etc/exports:

/mnt/myiso 192.168.0.0/255.255.255.0(async, no_subtree_check, ro, no_squash_root)

I have specified a range of IP addresses to export the directory to all the machines within the subnet. Only one IP address or a different IP address range can also specified there. Note that there should be NO space between the IP address/IP address range and the opening paranthesis that follows it. To learn more about the different options and their meanings that can be specified in the line, head on to this.

Start the nfs and portmap daemons and add them to the default runlevel(so that they start automatically at the boot time).

sh# /etc/init.d/nfs start
sh# rc-update add nfs default
sh# /etc/init.d/portmap start
sh# rc-update add portmap default

If you ever make changes to the /etc/exportfs file, then export all your changes using:

sh# exportfs -ra
sh# /etc/init.d/nfs reload
sh# exportfs

The last command shows you all the directories that are currently exported.

Now go to the other machine(Debian in my case) and use a simple mount command to access the exported directory.

sh# mkdir /mnt/remote
sh# mount 192.168.0.7:/mnt/myiso /mnt/remote
sh# ls /mnt/remote

You should see all the files from the exported directory. The IP address above is of the host system(Gentoo in my case) which is running the nfs daemon. Now you can use the /mnt/remote directory as any other directory on the local gnu/linux system. If the exported directory were a linux partition or a Windows FAT partition, then the write options could also have been ebabled by using the “rw” option in the /etc/exportfs file in place of the “ro” option. The following line can be added to the /etc/fstab file to mount this directory automatically at the boot time:

192.168.0.7:/mnt/myiso /mnt/remote nfs ro 0 0

The IP address, as above, is of the host system. You may be interested in this: NFS in Gentoo

Direct Server to Server Copy using FXP

2

FTP is still the best way to upload or download the files/folders from a website. I use gFTP to upload the files to my web space and its quite efficient at the job. Even if the connection gets interrupted in the middle of a transfer, it automatically reconnects and completes the job. It has many other useful features too. One feature that I thought was lacking in it is the ability to move a folder from one location to the other within the same server. Other people had complained for its lack of ability to move a file from one server to another server without involving the local machine in the process. Though many FTP clients do support such a feature(including gFTP, as I discovered later), I started searching for a dedicated FXP client. But what is FXP? From Wikipedia:

File eXchange Protocol (FXP) is a method of data transfer which uses the FTP protocol to transfer data from one remote server to another without routing this data through the client’s connection.

Note that FXP transfers need relevant support on the server side too(they must support PASV mode and PORT commands) – otherwise the FXP client can’t do anything about it. To install a FXP client on Gentoo:

  1. Add the following line to the file /etc/portage/package.keywords :
    >=net-ftp/gtkfxp-0.5 ~x86
  2. Emerge the package and run it:

    sh# emerge gtkfxp
    sh# gtkfxp &

Click the Connect button on the left side and fill the settings of the first server in the popped up dialog box, then click the Connect button on the right side and fill out the settings of the second server. You can now start transferring the files in between these two servers without involving your local machine. You can enter the information about the same server in both the dialog boxes if you want to transfer the files within the same server. Note that this software is buggy and very crash prone. gFTP too supports FXP transfers but it is seriously lacking in the documentation. I recently got an email from Chris explaining how to do FXP transfers in gFTP. The procedure described was like this:

Here’s how to use FXP wit gFTP: First, FXP requires you to connect to both sites
using the normal client. To do so please use Menu “FTP” -> “Window 1″ and then
select one of your bookmarked sites, and wait till it has properly connected.
You will see the file listing appear in window 1, which is the left panel.
Switch to “Window 2″ using “FTP” -> “Window 2″ and then connect via bookmark
to another (different!) site. That’s it. You can now transfer between them.

Thanks Chris for the great tip!

Go to Top