Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
Compiler Tools | ||||||||
Line: 32 to 32 | ||||||||
If you've worked on more than one system on the Nevis Linux cluster, you may have encountered the following problem: You compile a program | ||||||||
Changed: | ||||||||
< < | on a system with one version of Fedora, but find that there are | |||||||
> > | on a system with one version of Scientific Linux, but find that there are | |||||||
library incompatibilities when you try to execute the program on another version, or have problems linking with one of the ROOT or Geant4 libraries. | ||||||||
Line: 44 to 44 | ||||||||
libraries) on the applications server. I refer to them as the "standardized" compilers, since the compilers and the programs they produce should execute in the same way on all the Nevis systems. | ||||||||
Changed: | ||||||||
< < | The available versions are accessible via the setup command: | |||||||
> > | The available versions are accessible via the command: | |||||||
Changed: | ||||||||
< < | setup gcc34 # GCC 3.4.6 setup gcc44 # GCC 4.4.0 setup gcc45 # GCC 4.5.2 setup gcc47 # GCC 4.7.2 | |||||||
> > | module avail gcc | |||||||
Deleted: | ||||||||
< < | The command unsetup compiler will unset any variables associated
with the above setup commands; you'll then use the native compiler on
the system on which you're working.
If you just type setup , you'll see that I've prepared versions of CLHEP, Geant4, and ROOT compiled with the standardized versions of GCC. I'll generally stick to the main compiler version used by the current version of Scientific Linux![]() | |||||||
This is not a cure-all to the issue of mixed library versions on different systems. but it should go some way towards developing programs that will execute properly on all the systems in the batch farm. Note that some software development environments, e.g., LArSoft, come with their own standardized compilers. Do not use the Nevis compilers with those frameworks, unless you know what you're doing. | ||||||||
Deleted: | ||||||||
< < |
Distributed compilationDistcc![]() distcc will distribute successive compilations onto
different machines on the Nevis cluster. The typical use of distcc
might be:
make -j10 CXX="distcc g++"This assumes that the make ![]() Makefile , makefile , GNUmakefile ) has
been set up such that CXX is an alias for your C++ compiler.
The -j option specifies the number of simultaneous compilations that
the make![]() -j100 is
not unreasonable for our cluster (but see the warning below.)
Note that distcc does not distribute the pre-processing of the source
code (e.g., substituting #include files) or program linking. Only the
compilation process itself is distributed.
The above example command is actually quite risky on the Nevis
cluster. Our systems have different versions of Fedora
Core installed, and therefore different versions of the GNU C++
compiler. The simplest solution is to use the standardized compilers.
So the example command above could become:
make -j50 CXX="distcc ${GCC_DIR}/bin/g++"Another compilation tool I've installed on the Nevis Linux cluster is ccache ![]() ccache is in projects for which you find yourself
typing make clean;make frequently. With ccache , make clean is not
needed; instead you might type:
make CXX="ccache g++"The two compiler tools distcc and ccache can be used together to greatly speed up the compilation of large projects. To combine the two examples above, one would type:
make -j50 CXX="ccache distcc ${GCC_DIR}/bin/g++"To (hopefully) simplify the use of these tools with the standardized compilers I've prepared, I define the following variables when you use the setup commands described above: distgcc # ccache distcc ${GCC_DIR}/bin/gcc, for the C compiler distgpp # ccache distcc ${GCC_DIR}/bin/g++, for the C++ compiler distgxx # ccache distcc ${GCC_DIR}/bin/g++, for the C++ compiler(A variable named $distg++ would create problems in most shells.)
So a shorter make command one could use when I compile big projects is:
make -j50 CXX=$distgppFor my own work, I go further and define the following in my ~/.myprofile setup script:
# Make sure I use the distcc versions of the compiler. export CXX=${distgpp} export CC=${distgcc}So the command I'd actually use with those variables defined is: make -j50These tools can speed up the compilation of large projects by a factor of 5 to 10. A warning about distributed compilationA value of-j50 (which tells make to run 50 compilation processes at once) will work well with distcc if the build process has been organized in the following way:
![]()
![]() distcc is:
![]() -j10 or even -j5
may give the maximal compilation speed; any larger value and the
computer will bog down with dozens of pre-processing or linking processes. |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
Compiler Tools | ||||||||
Line: 6 to 6 | ||||||||
On this page:
| ||||||||
Changed: | ||||||||
< < | A discussion of compiler tools available at Nevis: | |||||||
> > | A discussion of compiler tools available at Nevis. | |||||||
Code checking | ||||||||
Changed: | ||||||||
< < | Before you compile, you can use the cppcheck![]() | |||||||
> > | The cppcheck command can find problems the compiler does not detect.
A typical use might be: | |||||||
cppcheck *.cxxThis will check all the .cxx files within the current directory. | ||||||||
Added: | ||||||||
> > | A more practical example:
cppcheck --enable=style myCodeDirThis will check all the C++ files in the sub-directory myCodeDir. | |||||||
Standardized compilersIf you've worked on more than one system on the Nevis Linux cluster, | ||||||||
Line: 35 to 47 | ||||||||
The available versions are accessible via the setup command:
setup gcc34 # GCC 3.4.6 | ||||||||
Deleted: | ||||||||
< < | setup gcc41 # GCC 4.1.1 | |||||||
setup gcc44 # GCC 4.4.0 | ||||||||
Added: | ||||||||
> > | setup gcc45 # GCC 4.5.2 setup gcc47 # GCC 4.7.2 | |||||||
The command unsetup compiler will unset any variables associated
with the above setup commands; you'll then use the native compiler on
the system on which you're working. | ||||||||
Changed: | ||||||||
< < | If you just type setup , you'll see that I've prepared
versions of CLHEP, Geant4, and ROOT compiled with the standardized GCC 3.4.6.
I'll generally stick to the main compiler version used by the current
version of Scientific Linux![]() | |||||||
> > | If you just type setup , you'll see that I've prepared versions of CLHEP, Geant4, and ROOT compiled with the standardized versions of GCC. I'll generally stick to the main compiler version used by the current version of Scientific Linux![]() | |||||||
This is not a cure-all to the issue of mixed library versions on different systems. but it should go some way towards developing programs that will execute properly on all the systems in the batch farm. | ||||||||
Added: | ||||||||
> > | Note that some software development environments, e.g., LArSoft, come with their own standardized compilers. Do not use the Nevis compilers with those frameworks, unless you know what you're doing. | |||||||
Distributed compilationDistcc![]() |
Line: 1 to 1 | ||||||||
---|---|---|---|---|---|---|---|---|
Added: | ||||||||
> > |
Compiler ToolsOn this page:
Code checkingBefore you compile, you can use the cppcheck![]() cppcheck *.cxxThis will check all the .cxx files within the current directory. Standardized compilersIf you've worked on more than one system on the Nevis Linux cluster, you may have encountered the following problem: You compile a program on a system with one version of Fedora, but find that there are library incompatibilities when you try to execute the program on another version, or have problems linking with one of the ROOT or Geant4 libraries. In general, the problem is with the library libstdc++.so, but there can be other issues as well. This problem can be particularly acute if you're preparing a job to run on our condor batch system, since there are systems with different OS versions on the farm. To help deal with this problem, I![]() setup gcc34 # GCC 3.4.6 setup gcc41 # GCC 4.1.1 setup gcc44 # GCC 4.4.0The command unsetup compiler will unset any variables associated
with the above setup commands; you'll then use the native compiler on
the system on which you're working.
If you just type setup , you'll see that I've prepared
versions of CLHEP, Geant4, and ROOT compiled with the standardized GCC 3.4.6.
I'll generally stick to the main compiler version used by the current
version of Scientific Linux![]() Distributed compilationDistcc![]() distcc will distribute successive compilations onto
different machines on the Nevis cluster. The typical use of distcc
might be:
make -j10 CXX="distcc g++"This assumes that the make ![]() Makefile , makefile , GNUmakefile ) has
been set up such that CXX is an alias for your C++ compiler.
The -j option specifies the number of simultaneous compilations that
the make![]() -j100 is
not unreasonable for our cluster (but see the warning below.)
Note that distcc does not distribute the pre-processing of the source
code (e.g., substituting #include files) or program linking. Only the
compilation process itself is distributed.
The above example command is actually quite risky on the Nevis
cluster. Our systems have different versions of Fedora
Core installed, and therefore different versions of the GNU C++
compiler. The simplest solution is to use the standardized compilers.
So the example command above could become:
make -j50 CXX="distcc ${GCC_DIR}/bin/g++"Another compilation tool I've installed on the Nevis Linux cluster is ccache ![]() ccache is in projects for which you find yourself
typing make clean;make frequently. With ccache , make clean is not
needed; instead you might type:
make CXX="ccache g++"The two compiler tools distcc and ccache can be used together to greatly speed up the compilation of large projects. To combine the two examples above, one would type:
make -j50 CXX="ccache distcc ${GCC_DIR}/bin/g++"To (hopefully) simplify the use of these tools with the standardized compilers I've prepared, I define the following variables when you use the setup commands described above: distgcc # ccache distcc ${GCC_DIR}/bin/gcc, for the C compiler distgpp # ccache distcc ${GCC_DIR}/bin/g++, for the C++ compiler distgxx # ccache distcc ${GCC_DIR}/bin/g++, for the C++ compiler(A variable named $distg++ would create problems in most shells.)
So a shorter make command one could use when I compile big projects is:
make -j50 CXX=$distgppFor my own work, I go further and define the following in my ~/.myprofile setup script:
# Make sure I use the distcc versions of the compiler. export CXX=${distgpp} export CC=${distgcc}So the command I'd actually use with those variables defined is: make -j50These tools can speed up the compilation of large projects by a factor of 5 to 10. A warning about distributed compilationA value of-j50 (which tells make to run 50 compilation processes at once) will work well with distcc if the build process has been organized in the following way:
![]()
![]() distcc is:
![]() -j10 or even -j5
may give the maximal compilation speed; any larger value and the
computer will bog down with dozens of pre-processing or linking processes. |