How to create a local repository for ubuntu 14.04. Debian - Creating a local repository (apt-mirror). Creating an Authenticated Repository

6. Automation

6.1 Server part

To which we add the treasured line:

0 0 * * * /usr/local/bin/repo_update.sh
Now every day at 0:00 our script will do all the routine work for us.

6.2 Client part

Let's not forget to make it executable.
sudo chmod +x /usr/local/bin/system_upd.sh

Then we open cron:
sudo crontab -e

And add the line:
40 17 * * * /usr/local/bin/system_upd.sh

Now every day at 17:40 the system will poll our repository for updates and update if any are found.

Attention! When working with crontab You should not forget that after the lines with tasks there must be an empty line, which is indicated by the sign " # ".
p.s.: I apologize for the lack of attached images, but in this case I think their presence is simply inappropriate.

Why might you need local? repository programs? Let's say you have several workstations in your office without Internet access. But they need to somehow install and update the software. In this case, a local repository is created on the server (or other machine with Internet access), which is periodically updated. And the rest of the workstations take programs from there. It may also be useful in case of limited or not always accessible internet. Download the repository - and at any time you can install any available program. There is nothing complicated in creating a local repository; now I will describe everything in detail.

It is very advisable to place the local repository on a separate hard drive (at least, a partition), so that in the event of an accident you do not lose everything you have downloaded through back-breaking labor. Distribution repository Debian 8 for architecture amd64 weighs more 40 gigabytes. Accordingly, if you also need i386 packages (32 bit), then the volume will double. So, let's say your system has a separate HDD 500 GB, mounted in /media/repo. This is what we will proceed from. Let's start with Debian 8 Jessie.

There are several ways to create a local repository on Debian/Ubuntu. I will use the simplest and most time-tested utility - apt-mirror. Install:

sudo apt install apt-mirror apache2

Create a directory for the repository:

sudo mkdir /media/repo/debian

And service directories:

sudo mkdir -p /media/repo/debian/(mirror,var,skel)

Setting it up. Open the configuration file:

sudo nano /etc/apt/mirror.list

set base_path /media/repo/debian

# set var_path $base_path/var

# set defaultarch

set run_postmirror 0

# Service parameters, not
set nthreads 20
set_tilde 0
#

# Mirror with packages for amd64 jessie (stable) + sources
deb-amd64 http://mirror.yandex.ru/debian jessie main contrib non-free

# Mirror with security updates amd64 jessie (stable) + sources
deb-amd64 http://security.debian.org/jessie/updates main contrib non-free


deb-amd64 http://mirror.yandex.ru/debian jessie main/debian-installer




skip-clean http://mirror.yandex.ru/debian/dists/jessie/main/installer-amd64/

# Mirror with packages for i386 jessie (stable) + sources
deb-i386 http://mirror.yandex.ru/debian jessie main contrib non-free
deb-src http://mirror.yandex.ru/debian jessie main contrib non-free
# Mirror with security updates i386 jessie (stable) + sources
deb-i386 http://security.debian.org/jessie/updates main contrib non-free
deb-src http://security.debian.org/jessie/updates main contrib non-free
# Mirror required for network installation (udebs)
deb-i386 http://mirror.yandex.ru/debian jessie main/debian-installer
# Delete files not indexed in Release
clean http://mirror.yandex.ru/debian
clean http://security.debian.org
# Disable cleaning of the selected folder
skip-clean http://mirror.yandex.ru/debian/dists/jessie/main/installer-i386/

Save. Let's start downloading the repository:

sudo apt-mirror

After the index files are downloaded, Apt-Mirror will tell you how many packages you need to download (the volume will be very, very large). All you have to do is wait. The system will do the rest itself. To automatically synchronize and clean mirrors, you need to add a line to the settings cron and set the appropriate time. Official mirrors are updated every 6 hours: 3:00, 9:00, 15:00, 21:00. For example like this:

crontab -e

05 01 * * * apt-mirror >> /var/log/apt-mirror.log
05 03 * * * /media/repo/debian/var/clean.sh >> /var/log/apt-mirror.log

For correct operation it is necessary to add symbolic links "stable","testing", "unstable" on jessie, stretch, sid accordingly (if you have them). Example for jessie:

ln -s /media/repo/debian/mirror/mirror.yandex.ru/debian/dists/jessie /media/repo/debian/mirror/mirror.yandex.ru/debian/dists/jessie/stable

We have installed a web server Apache for good reason. We need it to distribute packages from our local repository over the network (local). First, you need to configure access to the repository. To do this, you need to create one symbolic link:

cd /media/repo/debian/
sudo ln -s /media/repo/debian/mirror/mirror.yandex.ru/debian debian

Now on the client machine (which needs access to the local repository), provide the repository address. If the computer with the repository has a network name (for example server), then indicate it. Otherwise, indicate it by address IP address:

sudo nano /etc/apt/sources.list

deb http://server/debian jessie main contrib non-free
deb-src http://server/debian jessie main contrib non-free
deb http://server/debian jessie/updates main contrib non-free

If you specified loading 32-bit packages (i386) in the config, then do not forget to add this architecture to the system:

sudo dpkg --add-architecture i386

And update the package list:

sudo apt-get update

Then everything is as usual. For Ubuntu, everything is the same, except for the names of the repositories and the addition of a 32-bit architecture to a 64-bit system (not necessary). Config mirror.list for Ubuntu 14.04:

############# config ###################
# Base directory, where a local mirror of the Debian repository will be created
set base_path /media/repo/ubuntu

# Paths to mirror files, temporary files and apt-mirror execution log
# set mirror_path $base_path/mirror
# set skel_path $base_path/skel
# set var_path $base_path/var
# set cleanscript $var_path/clean.sh

# The architecture for which the mirror is created. Default is the architecture
# the system on which apt-mirror runs (amd64,i386 or others). No need to change
# because Below we will explicitly indicate both the mirror and the architecture.
# set defaultarch

# The path to the post-processing script is missing by default, but we don’t need it yet.
# set postmirror_script $var_path/postmirror.sh

# Do not run post-processing script
set run_postmirror 0

# Service parameters, not
set nthreads 20
set_tilde 0
#
############# end config ##############

deb-amd64 http://archive.ubuntu.com/ubuntu trusty main restricted
deb-amd64 http://archive.ubuntu.com/ubuntu trusty-updates main restricted
deb-amd64 http://archive.ubuntu.com/ubuntu trusty universe
deb-amd64 http://archive.ubuntu.com/ubuntu trusty-updates universe
deb-amd64 http://archive.ubuntu.com/ubuntu trusty multiverse
deb-amd64 http://archive.ubuntu.com/ubuntu trusty-updates multiverse
deb-amd64 http://archive.ubuntu.com/ubuntu trusty-security main restricted
deb-amd64 http://archive.ubuntu.com/ubuntu trusty-security universe
deb-amd64 http://archive.ubuntu.com/ubuntu trusty-security multiverse

deb-i386 http://archive.ubuntu.com/ubuntu trusty main restricted
deb-i386 http://archive.ubuntu.com/ubuntu trusty-updates main restricted
deb-i386 http://archive.ubuntu.com/ubuntu trusty universe
deb-i386 http://archive.ubuntu.com/ubuntu trusty-updates universe
deb-i386 http://archive.ubuntu.com/ubuntu trusty multiverse
deb-i386 http://archive.ubuntu.com/ubuntu trusty-updates multiverse
deb-i386 http://archive.ubuntu.com/ubuntu trusty-security main restricted
deb-i386 http://archive.ubuntu.com/ubuntu trusty-security universe
deb-i386 http://archive.ubuntu.com/ubuntu trusty-security multiverse

Well, accordingly, you need to change the symbolic link:

ln -s /media/repo/ubuntu/mirror/archive.ubuntu.com/debian/dists/trusdy /media/repo/ubuntu/mirror/archive.ubuntu.com/ubuntu/dists/trusty/stable

That's all. This is the simplest and, in my opinion, correct way to create a local repository on these systems. If you have any questions, write in the comments.

Prehistory
I recently thought about what would happen if my system suddenly refused to boot and I didn’t have enough intelligence or time to restore it. Then everything will have to start from scratch. But what about those packages that were faithfully downloaded throughout the year? What if there is not just one computer, but a whole network?

Everyone doesn’t want to download 800 meters of updates at all. For a long time I used the APTonCD utility, which suited me for the time being. But there is one nuance: when we create a repository with this utility in iso image(and/or we write it on a blank), then there is no way to add one or two packages to the repository without rebuilding the entire image. And if you take into account the fact that in the local package cache (from which this utility creates images) they are not stored forever and are deleted over time, then you have to store all the image files or risk losing important packages that were downloaded relatively long ago (since they are no longer in the cache). As a result, a large number of images are assembled, in which the packages are mostly repeated. There is only one way out. Create a full-fledged local repository, which is what we will do.

Preparation
We only need to install one package: apt-move. Many will say that you can do without it and they will be right, but I was looking for a simple way to create the “correct” repository. Therefore we do:
sudo apt-get install apt-move
Now we need to decide on the folder where our repository will be located. I chose /home//mirror . Although the name is not very good, since our repository is not a complete mirror of the official Ubuntu repository, but only a part of it that contains packages that have been downloaded at least once, but I’m too lazy to redo the script. So let it be like this:

Mkdir ~/mirror
cd ~/mirror

Now you're ready to move on to next step- at a construction site.

Settings
First, we need to configure the apt-move utility itself. To do this, let's do:
gksu gedit /etc/apt-move.conf
An editor with a settings file will open. Change the value of the LOCALDIR variable to LOCALDIR=/home//mirror (it is advisable to write the full absolute path, it’s more reliable). We also change PKGCOMP to PKGCOMP=none. This is compression. The fact is that I discovered glitches while using apt-move compression, so we will compress everything in our script without using this apt-move function. That's it, save and close. Now let's do it
gedit ~/mirror/create_repo.sh
We will write our future script into this file. Let's start:

#!/bin/bash
sudo apt-move get
sudo apt-move move

I won’t comment on this in detail, I’ll just say that here we check the packages in the local cache and copy them to the local repo, adhering to the official folder structure. (that's why we needed the apt-move package). Next we enter:

Prefix=/home/ /mirror
cd $prefix

Mkdir -p $prefix/pool/main
mkdir -p $prefix/pool/partner
mkdir -p $prefix/pool/non-free

Mkdir -p $prefix/dists/stable/main/binary-i386
mkdir -p $prefix/dists/stable/partner/binary-i386
mkdir -p $prefix/dists/stable/non-free/binary-i386

We check the presence of the necessary folders. We set the prefix variable in accordance with our location of the repository root. A caveat needs to be made here. It is possible that while working and updating your repository, you will have other components in the /pool folder (there are only three here: main partner non-free), then you will need to add lines by analogy with a simple copy-paste and replace them with the appropriate names. It's simple... Let's move on:

Dir=dists/stable/main/binary-i386
apt-ftparchive packages pool/main > $dir/Packages
gzip -9c<$dir/Packages >$dir/Packages.gz
bzip2 -9c<$dir/Packages >$dir/Packages.bz2
apt-ftparchive release $dir > $dir/Release
cat > $dir/Release<< EOF
Archive: stable
Suite: stable
Component: main
Origin:APT-Move
Label: APT-Move
Architecture: i386
EOF

We create so-called index files with a description of all packages that are included in a given distribution branch. (here this is main) We also create a Release file, which contains a description of this branch. In total, four files are created:
Packages
Packages.bz2
Packages.gz
Release
the Packages file itself and the accompanying Release file. You need to copy and paste this part of the code as many times as there are branches in your repository (I have three, so I copy two more times and change the corresponding names to non-free and partner). Next we add:

Dir=$prefix/dists/stable/
cat > $dir/Release<< EOF
Origin:APT-Move
Label: APT-Move
Suite: stable
Codename: unknown
EOF

Echo Date: `date -u +"%a, %d %b %Y %T UTC"` >> $dir/Release
cat >> $dir/Release<< EOF
Architectures: i386
Components: main non-free partner
Description: unknown
EOF

Apt-ftparchive release $dir >> $dir/Release

Here we create a Release file for the entire distribution (ours is stable) as a whole, which, in addition to other information, will contain checksums all Packages (and Release) files in all branches listed in Components: (as you can see, I still have three of them :)). Therefore, if you have other branches, be sure to edit this line. We save the script after executing the command

You can already create a repository and update from it. But the fact is that untrusted repositories have lower priority than trusted ones (or they are also called signed ones). Therefore, we will give our repository the same priority as the official one. For this we need a pgp key. If you don’t have one yet, you can create it using the seahorse program (Programs-Standards-Passwords and Encryption Keys). You can, of course, run the gpg program from the console, but the GUI is somehow closer to me. We create a key and now we need to export it so that clients can add it to the trusted ones. To do this, let's do

Cd ~/mirror gpg --export<имя_вашего_ключа>> apt.key

Apt-ftparchive release $dir >> $dir/Release zenity --entry \ --title="Key password" \ --text="Введите Ваш пароль:" \ --entry-text "" \ --hide-text | gpg -abs --yes --passphrase-fd 0 -o $dir/Release.gpg $dir/Release!}

As you can see, the zenity and gpg utility are used here. The first is a regular “decoration”. Well, I don't like sad people command line in English. language :), so here a graphical window is called up asking you to enter the password for your key. The second one directly signs our repository and it becomes trusted. That's all, save and close. If you have not yet given the right to execute to our script, then give it:

Sudo chmod +x ~/mirror/create_repo.sh
True, I haven’t stopped there yet. Since I need to update the repository from the Internet at work, and update the system from the repository at home, I decided to pack the entire local repository into one file and unpack it at home. So I added the following:

Zenity --question \ --title="Archiving" \ --text="Создать архив репозитария?" if [ "$?" = 0 ] ; then cd $prefix tar czf ../repo_`date +"%d-%m-%Y_%H-%M-%S"`.tar.gz ./ zenity --info --text="Архивирование завершено!" fi!}

Again, we use zenity to display graphic windows and, if the user agrees, we make an archive of the current turnip into the user’s folder.

Usage
In order to connect our repository, you need to add it to the /etc/apt/sources.list file. We do:

Gksu gedit /etc/apt/sources.list
and add the following line to the beginning of the file

Deb file:/home/ /mirror stable main non-free partner #Local repository
If you have other branches besides main non-free partner, you need to remember to add them. In addition, we need to add the public key of the repository with which we signed our rep. To do this you need to do

Sudo apt-key add apt.key

That's it, now we update the list of packages

Sudo apt-get update

P.S. Anyone who still gets a package from the Internet that is in the local repository, read this topic http://forum.runtu.org/index.php/topic,5403.msg44916.html#msg44916
P.P.S. Can someone help with a description of the Approx caching proxy server. It would be nice to use it here.

In this article we will look at how to create a new empty Git repository. We will create a local repository and also look at how to create a remote repository using Github as an example.

How to create a new empty repository

Create an empty directory for your future repository and go to it:

Mkdir myproject cd myproject

Now, to create a new repository in our directory, run the command:

Git init

This will create a new local empty repository. A message like this will be displayed on the screen:

Initialized empty Git repository in /path/to/myproject/.git/

In the directory myproject will appear hidden folder .git. You can see it by running ls -al

How to create a repository from existing files

Let's assume that you already have a project for which you want to create a Git repository. Creating a repository is done exactly as described above. Go to your project folder:

Cd myproject

Create a repository:

Git init

Now you can add all files to the index and make the first commit:

Git add -A git commit -m "First commit."

How to create a remote repository (using Github as an example)

You have created a local repository, now, for example, you need to add it to Github, thereby effectively creating a remote repository.

Go to https://githib.com and log in to your account. Click the New repository button. On the page that opens, enter the name of the repository ( Repository name) and press the button Create repository.

In your local repository now run the command:

Git remote add origin https://github.com/username/myproject.git

This command will add a remote repository named origin that points to your Github repository. So far we have only added an entry about the remote repository.

You can now run the git push command to push all your changes to the remote repository:

Git push -u origin master

You will need to enter your Github account login and password. The output of the command will be something like this:

$ git push -u origin master Username for "https://github.com": [email protected] Password for "https:// [email protected]@github.com": Counting objects: 4, done. Delta compression using up to 4 threads. Compressing objects: 100% (2/2), done. Writing objects: 100% (4/4), 252 bytes | 252.00 KiB /s, done. Total 4 (delta 0), reused 0 (delta 0) remote: remote: Create a pull request for "master" on GitHub by visiting: remote: https://github.com/username/myproject/pull /new/master remote: To https://github.com/username/myproject.git * master -> master Branch "master" set up to track remote branch "master" from "origin".

In the git push command we used the -u switch. This key is used to link the local branch master from remote origin/master(in our case, the remote branch did not exist, it was automatically created). Since the link is established, subsequent git pushes from the master branch can be done without specifying branches. That is, instead of git push origin master ), you can simply run the git push command.