より多くのメモリを備えたDockerfileからのDockerビルド
より多くのメモリを備えたDockerfileからdocker buildする方法は？
これはこれとは別の質問です dockerがDockerfileをビルドするときにより多くのメモリを許可する
ソフトウェアをネイティブにインストールする場合、十分なメモリがあり、正常にビルドおよびインストールできます marian tool
しかし、Dockerfilehttps://github.com/marian-nmt/marian/blob/master/scripts/docker/Dockerfile.cp を使用してDockerイメージをビルドすると、次のように失敗します複数のメモリを使い果たしたエラー
virtual memory exhausted: Cannot allocate memory
[アウト]：
Step : RUN cmake $MARIANPATH && make -j
---> Running in 4867d166d17a
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CUDA_TOOLKIT_ROOT_DIR not found or specified
-- Cannot find CUDA libraries. Compiling without them.
-- Could NOT find CUDA (missing: CUDA_TOOLKIT_ROOT_DIR CUDA_NVCC_EXECUTABLE CUDA_INCLUDE_DIRS CUDA_CUDART_LIBRARY)
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Boost version: 1.58.0
-- Found the following Boost libraries:
-- system
-- filesystem
-- program_options
-- timer
-- iostreams
-- python
-- thread
-- chrono
-- regex
-- date_time
-- atomic
-- Found Python
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython2.7.so (found suitable version "2.7.12", minimum required is "2.7")
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.8")
-- Found Git: /usr/bin/git (found version "2.7.4")
-- Git version: 5abc774
-- Found SparseHash: /usr/include
-- Configuring done
-- Generating done
-- Build files have been written to: /marian/build
Scanning dependencies of target fast_align
Scanning dependencies of target extract_Lex
Scanning dependencies of target libcnpy
Scanning dependencies of target atools
Scanning dependencies of target libyaml-cpp-amun
[ 2%] Building CXX object src/amun/3rd_party/extract_Lex/CMakeFiles/extract_Lex.dir/exception.cpp.o
[ 2%] Building CXX object src/amun/3rd_party/extract_Lex/CMakeFiles/extract_Lex.dir/utils.cpp.o
[ 3%] Building CXX object src/amun/3rd_party/fast_align/CMakeFiles/fast_align.dir/src/ttables.cc.o
[ 6%] Building CXX object src/amun/3rd_party/fast_align/CMakeFiles/fast_align.dir/src/fast_align.cc.o
[ 7%] Building CXX object src/amun/3rd_party/extract_Lex/CMakeFiles/extract_Lex.dir/extract-Lex-main.cpp.o
[ 10%] Building CXX object src/amun/3rd_party/CMakeFiles/libcnpy.dir/cnpy/cnpy.cpp.o
[ 10%] Building CXX object src/amun/3rd_party/fast_align/CMakeFiles/atools.dir/src/atools.cc.o
[ 10%] Building CXX object src/amun/3rd_party/fast_align/CMakeFiles/atools.dir/src/alignment_io.cc.o
[ 11%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/regex_yaml.cpp.o
[ 12%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/scanner.cpp.o
[ 14%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/convert.cpp.o
[ 15%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/stream.cpp.o
[ 16%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/memory.cpp.o
[ 17%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/node.cpp.o
[ 19%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/directives.cpp.o
[ 20%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/null.cpp.o
[ 21%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/emitfromevents.cpp.o
[ 23%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/singledocparser.cpp.o
[ 24%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/emitterstate.cpp.o
[ 25%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/scantag.cpp.o
[ 26%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/simplekey.cpp.o
[ 28%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/binary.cpp.o
[ 29%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/parser.cpp.o
[ 30%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/nodeevents.cpp.o
[ 32%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/emit.cpp.o
[ 33%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/exp.cpp.o
[ 35%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/nodebuilder.cpp.o
[ 38%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/scanscalar.cpp.o
[ 39%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/emitter.cpp.o
[ 41%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/tag.cpp.o
[ 42%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/scantoken.cpp.o
[ 43%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/ostream_wrapper.cpp.o
Scanning dependencies of target libcommon
[ 37%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/node_data.cpp.o
[ 44%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/contrib/graphbuilder.cpp.o
[ 46%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/emitterutils.cpp.o
[ 34%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/contrib/graphbuilderadapter.cpp.o
[ 47%] Building CXX object src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/parse.cpp.o
[ 48%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/scorer.cpp.o
[ 50%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/git_version.cpp.o
[ 51%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/base_matrix.cpp.o
[ 52%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/sentence.cpp.o
[ 53%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/config.cpp.o
[ 55%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/utils.cpp.o
[ 56%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/vocab.cpp.o
[ 65%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/filter.cpp.o
[ 66%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/god.cpp.o
[ 67%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/exception.cpp.o
[ 69%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/search.cpp.o
[ 69%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/hypothesis.cpp.o
[ 70%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/translation_task.cpp.o
[ 70%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/printer.cpp.o
[ 70%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/sentences.cpp.o
[ 70%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/history.cpp.o
[ 70%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/logging.cpp.o
[ 70%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/processor/bpe.cpp.o
[ 71%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/output_collector.cpp.o
[ 73%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/types.cpp.o
[ 74%] Building CXX object src/amun/CMakeFiles/libcommon.dir/common/loader.cpp.o
virtual memory exhausted: Cannot allocate memory
virtual memory exhausted: Cannot allocate memory
virtual memory exhausted: Cannot allocate memory
そして最終的には次のようになります：
src/amun/CMakeFiles/libcommon.dir/build.make:254: recipe for target 'src/amun/CMakeFiles/libcommon.dir/common/loader.cpp.o' failed
virtual memory exhausted: Cannot allocate memory
make[2]: *** [src/amun/CMakeFiles/cpumode.dir/cpu/decoder/encoder_decoder.cpp.o] Error 1
src/amun/CMakeFiles/cpumode.dir/build.make:110: recipe for target 'src/amun/CMakeFiles/cpumode.dir/cpu/decoder/encoder_decoder.cpp.o' failed
[ 79%] Built target libcnpy
virtual memory exhausted: Cannot allocate memory
src/amun/CMakeFiles/libcommon.dir/build.make:326: recipe for target 'src/amun/CMakeFiles/libcommon.dir/common/printer.cpp.o' failed
make[2]: *** [src/amun/CMakeFiles/libcommon.dir/common/printer.cpp.o] Error 1
CMakeFiles/Makefile2:340: recipe for target 'src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/all' failed
make[1]: *** [src/amun/3rd_party/yaml-cpp/CMakeFiles/libyaml-cpp-amun.dir/all] Error 2
CMakeFiles/Makefile2:182: recipe for target 'src/amun/CMakeFiles/libcommon.dir/all' failed
make[1]: *** [src/amun/CMakeFiles/libcommon.dir/all] Error 2
make[1]: *** [src/amun/CMakeFiles/cpumode.dir/all] Error 2
CMakeFiles/Makefile2:110: recipe for target 'src/amun/CMakeFiles/cpumode.dir/all' failed
make: *** [all] Error 2
docker build --helpを見ると、メモリ使用量を制御するオプションがあります。
$ docker build --help
Usage: docker build [OPTIONS] PATH | URL | -
Build an image from a Dockerfile
Options:
--build-arg list Set build-time variables (default [])
--cache-from stringSlice Images to consider as cache sources
--cgroup-parent string Optional parent cgroup for the container
--compress Compress the build context using gzip
--cpu-period int Limit the CPU CFS (Completely Fair Scheduler) period
--cpu-quota int Limit the CPU CFS (Completely Fair Scheduler) quota
-c, --cpu-shares int CPU shares (relative weight)
--cpuset-cpus string CPUs in which to allow execution (0-3, 0,1)
--cpuset-mems string MEMs in which to allow execution (0-3, 0,1)
--disable-content-trust Skip image verification (default true)
-f, --file string Name of the Dockerfile (Default is 'PATH/Dockerfile')
--force-rm Always remove intermediate containers
--help Print usage
--isolation string Container isolation technology
--label list Set metadata for an image (default [])
-m, --memory string Memory limit
--memory-swap string Swap limit equal to memory plus swap: '-1' to enable unlimited swap
--network string Set the networking mode for the RUN instructions during build (default "default")
--no-cache Do not use cache when building the image
--pull Always attempt to pull a newer version of the image
-q, --quiet Suppress the build output and print image ID on success
--rm Remove intermediate containers after a successful build (default true)
--security-opt stringSlice Security options
--shm-size string Size of /dev/shm, default value is 64MB
-t, --tag list Name and optionally a tag in the 'name:tag' format (default [])
--ulimit ulimit Ulimit options (default [])
しかし、-mオプションを正確に配置する場所の正しい構文を理解できませんでした-_- |||
Dockerfileの前ではありません：
# Before Docker file.
$ docker build -m 4g Dockerfile.cpu -t ibot-cpu .
"docker build" requires exactly 1 argument(s).
See 'docker build --help'.
Usage: docker build [OPTIONS] PATH | URL | -
Build an image from a Dockerfile
-tの前のDockerfileの後ではありません
# Before -t
$ docker build Dockerfile.cpu -m 4g -t ibot-cpu .
"docker build" requires exactly 1 argument(s).
See 'docker build --help'.
Usage: docker build [OPTIONS] PATH | URL | -
Build an image from a Dockerfile
ローカルパスの前の-tの後ではない
# Before local path
$ docker build Dockerfile.cpu -t ibot-cpu -m 4g .
"docker build" requires exactly 1 argument(s).
See 'docker build --help'.
Usage: docker build [OPTIONS] PATH | URL | -
Build an image from a Dockerfile
最後のローカルパスの後でもありません...
# At the end...
$ docker build Dockerfile.cpu -t ibot-cpu . -m 4g
"docker build" requires exactly 1 argument(s).
See 'docker build --help'.
Usage: docker build [OPTIONS] PATH | URL | -
Build an image from a Dockerfile
より多くのメモリを備えたDockerfileからdocker buildする方法は？
私のドッカーのバージョン：
docker version
Client:
Version: 17.03.1-ce
API version: 1.27
Go version: go1.7.5
Git commit: c6d412e
Built: Tue Mar 28 00:40:02 2017
OS/Arch: darwin/AMD64
Server:
Version: 17.04.0-ce
API version: 1.28 (minimum version 1.12)
Go version: go1.7.5
Git commit: 4845c56
Built: Wed Apr 5 18:45:47 2017
OS/Arch: linux/AMD64
Experimental: false
それは秩序に関するものではありません。 Dockerfileは-fで指定する必要があります
docker build -f Dockerfile.cpu -t ibot-cpu -m 4g .
あなたがヘルプで見ることができるように。
ただし、デフォルトでは、Dockerはコンテナーのメモリを制限しないことに注意してください。それは全体の空きメモリを取ることができます。
私が見ることができるように、あなたはOSX上にいて、Linux VM上でDockerを実行しています。タスクバーのクジラのアイコンをクリックして最大メモリを構成します。デフォルトは2Gです。
詳細については、他の回答を参照してください： Dockerコンテナーにより多くのメモリを割り当てる方法