1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306
|
// Copyright (c) 2018-2022, Sylabs Inc. All rights reserved.
// This software is licensed under a 3-clause BSD license. Please consult the
// LICENSE.md file distributed with the sources of this project regarding your
// rights to use or distribute this software.
package sources
import (
"bytes"
"context"
"fmt"
"io"
"os"
"os/exec"
"path/filepath"
"runtime"
"strings"
"syscall"
"time"
"github.com/sylabs/singularity/v4/internal/pkg/util/bin"
"github.com/sylabs/singularity/v4/internal/pkg/util/fs"
"github.com/sylabs/singularity/v4/pkg/build/types"
"github.com/sylabs/singularity/v4/pkg/sylog"
"github.com/sylabs/singularity/v4/pkg/util/namespaces"
)
// debootstrapArchs is a map of GO Archs to official Debian ports
// https://www.debian.org/ports/
var debootstrapArchs = map[string]string{
"386": "i386",
"amd64": "amd64",
"arm": "armhf",
"arm64": "arm64",
"ppc64le": "ppc64el",
"mipsle": "mipsel",
"mips64le": "mips64el",
"s390x": "s390x",
}
// DebootstrapConveyorPacker holds stuff that needs to be packed into the bundle
type DebootstrapConveyorPacker struct {
b *types.Bundle
mirrorurl string
osversion string
include string
}
// prepareFakerootEnv prepares a build environment to
// make fakeroot working with debootstrap.
func (cp *DebootstrapConveyorPacker) prepareFakerootEnv(ctx context.Context) (func(), error) {
truePath, err := bin.FindBin("true")
if err != nil {
return nil, fmt.Errorf("while searching true command: %s", err)
}
mountPath, err := bin.FindBin("mount")
if err != nil {
return nil, fmt.Errorf("while searching mount command: %s", err)
}
mknodPath, err := bin.FindBin("mknod")
if err != nil {
return nil, fmt.Errorf("while searching mknod command: %s", err)
}
procFsPath := "/proc/filesystems"
devs := []string{
"/dev/null",
"/dev/random",
"/dev/urandom",
"/dev/zero",
}
devPath := filepath.Join(cp.b.RootfsPath, "dev")
if err := os.Mkdir(devPath, 0o755); err != nil {
return nil, fmt.Errorf("while creating %s: %s", devPath, err)
}
innerCtx, cancel := context.WithCancel(ctx)
umountFn := func() {
cancel()
syscall.Unmount(mountPath, syscall.MNT_DETACH)
syscall.Unmount(mknodPath, syscall.MNT_DETACH)
for _, d := range devs {
path := filepath.Join(cp.b.RootfsPath, d)
syscall.Unmount(path, syscall.MNT_DETACH)
}
}
// bind /bin/true on top of mount/mknod command
// so debootstrap wouldn't fail while preparing
// chroot environment
if err := syscall.Mount(truePath, mountPath, "", syscall.MS_BIND, ""); err != nil {
return umountFn, fmt.Errorf("while mounting %s to %s: %s", truePath, mountPath, err)
}
if err := syscall.Mount(truePath, mknodPath, "", syscall.MS_BIND, ""); err != nil {
return umountFn, fmt.Errorf("while mounting %s to %s: %s", truePath, mknodPath, err)
}
// very dirty workaround to address issue with makedev
// package installation during ubuntu bootstrap, we watch
// for the creation of $ROOTFS/sbin/MAKEDEV and truncate
// the file to obtain an equivalent of /bin/true, for makedev
// post-configuration package we also need to create at least
// one /dev/ttyX file
go func() {
makedevPath := filepath.Join(cp.b.RootfsPath, "/sbin/MAKEDEV")
for {
select {
case <-innerCtx.Done():
break
case <-time.After(100 * time.Millisecond):
if _, err := os.Stat(makedevPath); err == nil {
os.Truncate(makedevPath, 0)
os.Create(filepath.Join(cp.b.RootfsPath, "/dev/tty1"))
break
}
}
}
}()
// debootstrap look at /proc/filesystems to check
// if sysfs is present, we bind /dev/null on top
// of /proc/filesystems to trick debootstrap to not
// mount /sys
if err := syscall.Mount("/dev/null", procFsPath, "", syscall.MS_BIND, ""); err != nil {
return umountFn, fmt.Errorf("while mounting /dev/null to %s: %s", procFsPath, err)
}
// mount required block devices
for _, p := range devs {
rootfsPath := filepath.Join(cp.b.RootfsPath, p)
if err := fs.Touch(rootfsPath); err != nil {
return umountFn, fmt.Errorf("while creating %s: %s", rootfsPath, err)
}
if err := syscall.Mount(p, rootfsPath, "", syscall.MS_BIND, ""); err != nil {
return umountFn, fmt.Errorf("while mounting %s to %s: %s", p, rootfsPath, err)
}
}
return umountFn, nil
}
// Get downloads container information from the specified source
func (cp *DebootstrapConveyorPacker) Get(ctx context.Context, b *types.Bundle) (err error) {
cp.b = b
// check for debootstrap on system(script using "singularity_which" not sure about its importance)
debootstrapPath, err := bin.FindBin("debootstrap")
if err != nil {
return fmt.Errorf("debootstrap is not in PATH... Perhaps 'apt-get install' it: %v", err)
}
if err = cp.getRecipeHeaderInfo(); err != nil {
return err
}
if os.Getuid() != 0 {
return fmt.Errorf("you must be root to build with debootstrap")
}
// Debian port arch values do not always match GOARCH values, so we need to look it up.
debArch, ok := debootstrapArchs[runtime.GOARCH]
if !ok {
return fmt.Errorf("Debian arch not known for GOARCH %s", runtime.GOARCH)
}
insideUserNs, setgroupsAllowed := namespaces.IsInsideUserNamespace(os.Getpid())
if insideUserNs && setgroupsAllowed {
umountFn, err := cp.prepareFakerootEnv(ctx)
if umountFn != nil {
defer umountFn()
}
if err != nil {
return fmt.Errorf("while preparing fakeroot build environment: %s", err)
}
}
// run debootstrap command
cmd := exec.Command(debootstrapPath, `--variant=minbase`, `--exclude=openssl,udev,debconf-i18n,e2fsprogs`, `--include=apt,`+cp.include, `--arch=`+debArch, cp.osversion, cp.b.RootfsPath, cp.mirrorurl)
sylog.Debugf("\n\tDebootstrap Path: %s\n\tIncludes: apt(default),%s\n\tDetected Arch: %s\n\tOSVersion: %s\n\tMirrorURL: %s\n", debootstrapPath, cp.include, runtime.GOARCH, cp.osversion, cp.mirrorurl)
// run debootstrap
out, err := cmd.CombinedOutput()
io.Copy(os.Stdout, bytes.NewReader(out))
if err != nil {
dumpLog := func(fn string) {
if _, err := os.Stat(fn); os.IsNotExist(err) {
return
}
fh, err := os.Open(fn)
if err != nil {
sylog.Debugf("Cannot open %s: %#v", fn, err)
return
}
defer fh.Close()
log, err := io.ReadAll(fh)
if err != nil {
sylog.Debugf("Cannot read %s: %#v", fn, err)
return
}
sylog.Debugf("Contents of %s:\n%s", fn, log)
}
dumpLog(filepath.Join(cp.b.RootfsPath, "debootstrap/debootstrap.log"))
return fmt.Errorf("while debootstrapping: %v", err)
}
return nil
}
// Pack puts relevant objects in a Bundle!
func (cp *DebootstrapConveyorPacker) Pack(context.Context) (*types.Bundle, error) {
// change root directory permissions to 0755
if err := os.Chmod(cp.b.RootfsPath, 0o755); err != nil {
return nil, fmt.Errorf("while changing bundle rootfs perms: %v", err)
}
err := cp.insertBaseEnv(cp.b)
if err != nil {
return nil, fmt.Errorf("while inserting base environment: %v", err)
}
err = cp.insertRunScript(cp.b)
if err != nil {
return nil, fmt.Errorf("while inserting runscript: %v", err)
}
return cp.b, nil
}
func (cp *DebootstrapConveyorPacker) getRecipeHeaderInfo() (err error) {
var ok bool
// get mirrorURL, OSVerison, and Includes components to definition
cp.mirrorurl, ok = cp.b.Recipe.Header["mirrorurl"]
if !ok {
return fmt.Errorf("invalid debootstrap header, no mirrorurl specified")
}
cp.osversion, ok = cp.b.Recipe.Header["osversion"]
if !ok {
return fmt.Errorf("invalid debootstrap header, no osversion specified")
}
include := cp.b.Recipe.Header["include"]
// check for include environment variable and add it to requires string
include += ` ` + os.Getenv("INCLUDE")
// trim leading and trailing whitespace
include = strings.TrimSpace(include)
// convert Requires string to comma separated list
cp.include = strings.Replace(include, ` `, `,`, -1)
return nil
}
func (cp *DebootstrapConveyorPacker) insertBaseEnv(b *types.Bundle) (err error) {
if err = makeBaseEnv(b.RootfsPath); err != nil {
return
}
return nil
}
func (cp *DebootstrapConveyorPacker) insertRunScript(b *types.Bundle) (err error) {
f, err := os.Create(b.RootfsPath + "/.singularity.d/runscript")
if err != nil {
return
}
defer f.Close()
_, err = f.WriteString("#!/bin/sh\n")
if err != nil {
return
}
if err != nil {
return
}
f.Sync()
err = os.Chmod(b.RootfsPath+"/.singularity.d/runscript", 0o755)
if err != nil {
return
}
return nil
}
// CleanUp removes any tmpfs owned by the conveyorPacker on the filesystem
func (cp *DebootstrapConveyorPacker) CleanUp() {
cp.b.Remove()
}
|