Dokumentace NGI NGI http://localhost/mediawiki/index.php/Main_Page MediaWiki 1.26.0 first-letter Média Speciální Diskuse Uživatel Diskuse s uživatelem Dokumentace NGI Diskuse k Dokumentace NGI Soubor Diskuse k souboru MediaWiki Diskuse k MediaWiki Šablona Diskuse k šabloně Nápověda Diskuse k nápovědě Kategorie Diskuse ke kategorii Gadget Gadget talk Gadget definition Gadget definition talk 0 144 1916 399 2016-04-30T07:40:29Z Root 1 wikitext text/x-wiki This is a general format for application description. If you would like to create [[:Category:Applications|a new application page]], please use this format. --------------------------------------------- = Description = = Availability = = Licence = = Use = = Documentation = = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = [[Category:Applications]] pmkcj2mgtook1ssd642iymj0d1cinii 0 146 5217 4963 2016-05-13T09:15:40Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Babel|Česká verze--> ===Description=== Program Babel serves to conversion between diferent formates of molecular structures. As input files is Babel available to recognize following formats: *Alchemy *AMBER PREP *Ball and Stick *Biosym .CAR *Boogie *Cacao Cartesian *Cambridge CADPAC* *CHARMm* *Chem3D Cartesian 1 *Chem3D Cartesian 2 *CSD CSSR *CSD FDAT* *CSD GSTAT *Free Form Fractional *GAMESS Output* *Gaussian Z-Matrix *Gaussian Output* *Hyperchem HIN* *Mac Molecule *Macromodel *Micro World *MM2 Input *MM2 Ouput *MM3* *MMADS* *MDL MOLfile* *MOLIN* *Mopac Cartesian *Mopac Internal *Mopac Output *PDB *ShelX *Sybyl Mol* *Sybyl Mol2* *XYZ As an output file can Babel write: *Alchemy *Ball and Stick *Cacao Cartesian *CAChe MolStruct* *Chem3D Cartesian 1 *Chem3D Cartesian 2 *ChemDraw Conn. Table* *CSD CSSR* *Gamess Input* *Gaussian Cartesian* *Gaussian Z-matrix *Hyperchem HIN* *IDATM *Mac Molecule *Macromodel *Micro World *MM2 Input* *MM2 Ouput *MM3* *MMADS* *MDL Molfile* *Mopac Cartesian *Mopac Internal *PDB *Report *Sybyl Mol* *Sybyl Mol2* *XYZ Babel is capable of assigning hybridization, bond order, and connectivity when these elements are not present in the input file. ===Availability=== Actually is version 1.1b5 available. ===Use=== Program Babel is available in module babel. Load modul ("module ad babel") and run "babel" or "babel - m" . ===Documentation=== [http://server.ccl.net/cca/software/UNIX/babel/README.BABEL.shtml On-line] documentation is available. ===Supported platforms=== Any UNIX version (AIX, Ultrix, Sun-OS, Convex, SGI), Cray, Linux, MS-DOS a Macs. ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://server.ccl.net/cca/software/UNIX/babel/README.BABEL.shtml ====Other information==== Possibility of try a program [http://openbabel.sourceforge.net/ Open Babel]. [[Kategorie:Applications]] ax0d2qio9n1eh3v4khof9lxvnf6nhkz 0 147 5153 5024 2016-05-13T09:14:40Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--BEAST|Česká verze--> ===Description=== BEAST is a cross-platform program for Bayesian MCMC analysis of molecular sequences. It is entirely orientated towards rooted, time-measured phylogenies inferred using strict or relaxed molecular clock models. It can be used as a method of reconstructing phylogenies but is also a framework for testing evolutionary hypotheses without conditioning on a single tree topology. BEAST uses MCMC to average over tree space, so that each tree is weighted proportional to its posterior probability. We include a simple to use user-interface program for setting up standard analyses and a suit of programs for analysing the results. ===Availability=== Program is actually available in version BEAST 1.6.1, 1.7.0, 1.7.4, 1.8.2 and 2.0.1 ===Use=== Program Beast is available in module beast. Load modul ("module ad beast" for 2.0.1 and "module ad beast-1.6.1" for 1.6.1) and run "beast" with option (see below) or "babel - help" for help . Usage: <source lang="bash">beast [-verbose] [-warnings] [-strict] [-window] [-options] [-working] [-seed] [-prefix <PREFIX>] [-overwrite] [-errors <i>] [-threads <i>] [-java] [-beagle] [-beagle_info] [-beagle_order <order>] [-beagle_instances <i>] [-beagle_CPU] [-beagle_GPU] [-beagle_SSE] [-beagle_single] [-beagle_double] [-beagle_scaling <default|none|dynamic|always>] [-help] [<input-file-name>] </source> For parallel running are required additional parameters -beagle_instances N -threads N where N is one less value than number of processors requested in PBS task. ===Documentation=== http://beast.bio.ed.ac.uk/ and http://beast2.org ===Licence=== The complete source code for BEAST is now distributed under the GNU Lesser General Public License and the source code, bug tracking database and information for developers is now hosted by Google: http://beast-mcmc.googlecode.com/ ===Supported platforms=== Linux 64bit machines ( x86_64 property) ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://beast.bio.ed.ac.uk/ [[Kategorie:Applications]] [[Kategorie:English]] n14o49za2zldb435y5rjd0hq6okqglt 0 151 5100 4951 2016-05-13T09:13:54Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Gromacs|Česká verze--> =Description= GROMACS is program package, which enables to define two basic tasks in the area of mechanics of molecules and dynamics: *1. Minimalization of energy of system. *2. Dynamic behaviour of molecular systems. Utilizing these basic approaches one can easily define tasks concerning system response at a molecular level. Mechanical response after deformation of system in equilibrium can serve as such example. Another possibility is drug design domain. To have an active drug it is neccessary that its structure has to be complemetary to the active side in a cell (i.e. key-lock approach). Diffusion can serve as an example of dynamic behaviour study. One can need to obtain several properties of a system as pressure, compresibility factor, heat capacity. It is neccessary to take into account two restrictions - real time of simulation (nanoseconds) and number of atoms (10.000). =Availability= * '''version 4.6.5''' ** <code>gromacs-4.6.5</code> &ndash; basic installation with support of GPU computing and PLUMED extension * '''version 4.6.1''' (thanks to O. Kroutil for compilation) &ndash; modules: ** <code>gromacs-4.6.1</code> ** <code>gromacs-4.6.1-parallel</code> (with MPI support) ** <code>gromacs-4.6.1-gpu</code> (with GPU-computing support) ** <code>gromacs-4.6.1-plumed</code> (GPU-computing and MPI support, compiled with [http://www.plumed-code.org/ PLUMED] extension) ** <code>gromacs-4.6.1-plumed_d</code> (GPU-computing and MPI support with ''double precision'', compiled with [http://www.plumed-code.org/ PLUMED] extension) * ''version 4.5.5'' -- module <code>gromacs-4.5.5</code> and module <code>gromacs-4.5.5-parallel</code> (MPI support) * ''version 4.0.7'' -- module <code>gromacs</code> and module <code>gromacs-parallel</code> (MPI support) =Use= {{MPI application}} Initialize environment by loading the appropriate module (see above) and run program. For basic command of Gromacs is needed two executable files grompp for compilation of the task and mdrun for running of the task. To this executable files are needed source files. File with axis of all athoms with suffix *.gro, program with topology with *.top suffix, in which are registrated linkage between each athom, program with *.mdp suffix, in which are written parameters of simulation. Example of task: <source lang="bash"> ./grompp –f ParametersOfSimulation.mdp –c Axis.gro –p Topology.top ./mdrun –s Topology.tpr -x Trajectory.xtc -o Trajectory.trr –e Energy.edr </source> Suffixes are not compulsory. In case you do not fill it, program will proceed automatically files called grompp.mdp, conf.gro, topol.top, topol.trr, traj.xtc, ener.edr. In enter directory must be files gro, top and mdp. If the program doesnt find them, reports error. Module gromacs makes access to defaut single-precision version and double-precison programs with _d suffix. Required MPI library is implemented in module gromacs-parallel. [[How to compute/Parallelization|Parallel version]] mdrun is also termed by suffix _mpi (_mpi_d in case double precision). It supports integration with PBS (it detects allocated nodes itself, in case of killing a job, it kills all processes at all machines) and automatic best choice of available connection (Infiniband or Myrinet, where available). So to run a job in PBS you need: <source lang="bash"> module add gromacs-parallel mpirun mdrun_mpi ... </source> Limitation for parallel calculations: If your system supports threading, mdrun will be compiled with the -nt option, which can then be used to execute parallel calculations. If you do not use -nt, then GROMACS will use the maximum number it thinks is available (see http://www.gromacs.org/Documentation/Terminology/Threading)! ===Using GPU=== <source lang="bash"> #!/bin/bash #PBS -l nodes=1:ppn=4:gpu=1 #PBS -q gpu@arien.ics.muni.cz #PBS -l mem=10g #PBS -j oe trap "clean_scratch" TERM EXIT # let's add the maple module module add gromacs-gpu # let's set the necessary variables DATADIR="$PBS_O_WORKDIR" # let's copy the input data cp $DATADIR/{pme_verlet.mdp,conf.gro,topol.top,topol.tpr} $SCRATCHDIR # let's change the working directory cd $SCRATCHDIR # let's perform the computation grompp -f pme_verlet.mdp -c conf.gro -p topol.top mdrun -nt $PBS_NUM_PPN -s topol.tpr # let's gzip and copy-out the result tar czf output.tgz ./* cp output.tgz $DATADIR || export CLEAN_SCRATCH=false </source> ===MPI parallelization notes=== ====MPI and OpenMP combination==== In some situations you can accelerate your computiation by combining MPI and OpenMP. See [http://manual.gromacs.org/online/mdrun.html more detailed documentation] for the exact cases. Use example: <source lang="bash"> mpirun -env OMP_NUM_THREADS $TORQUE_RESC_PROC -np $TORQUE_RESC_TOTAL_NODECT -rmk user -hosts $(echo $(sort -u < $PBS_NODEFILE) |sed 's/ /,/g') mdrun_mpi -s job.tpr -maxh 23. </source> ====Large scale computations==== If you need to run a large computation across many nodes, it's worth to use the least number of same machines than spread it across HW different nodes. Use the whole machines with <tt>#excl</tt> qsub option. It is also possible to tie up the threads with specific processor cores (option <tt>-pin on</tt>). In case of [http://metavo.metacentrum.cz/pbsmon2/props HyperThreading capable cluster] you can use the HT double core feature. For example: <source lang="bash"> $ qsub -l nodes=2:ppn=1:cl_luna#excl ... mpirun -env OMP_NUM_THREADS $((2*TORQUE_RESC_PROC)) -np $TORQUE_RESC_TOTAL_NODECT -rmk user -hosts $(echo $(sort -u < $PBS_NODEFILE) |sed 's/ /,/g') mdrun_mpi -pin on -s job.tpr -maxh 23. </source> or <source lang="bash"> mpirun -env OMP_NUM_THREADS 1 -np $((2*TORQUE_RESC_PROC*TORQUE_RESC_TOTAL_NODECT)) -rmk user -hosts $(echo $(sort -u < $PBS_NODEFILE) |sed 's/ /,/g') mdrun_mpi -pin on -s job.tpr -maxh 23. </source> =Documentation= Manual pages are avalaible on web page of software: http://www.gromacs.org/Documentation =Licence= Program is available under GPL licence. =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://www.gromacs.org/ =Known issues= New version of Gromacs 4.5.5 is using different naming conventions (e.g. use NA instead of NA+). [[Category:Applications]] [[Category:English]] 6syb806akyzsw21602ongs1sqdc23ou 0 152 5022 4867 2016-05-13T09:12:40Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Molden|Česká verze--> ===Description=== Molden is a package for displaying Molecular Density from the Ab Initio packages GAMESS-UK , GAMESS-US and GAUSSIAN and the Semi-Empirical packages Mopac/Ampac, Molden is capable of displaying Molecular Orbitals, the electron density and the Molecular minus Atomic density. Either the spherically averaged atomic density or the oriented ground state atomic density can be subtracted for a number of standard basis sets. Molden supports contour plots, 3-d grid plots with hidden lines and a combination of both. It also can animate reaction paths and molecular vibrations. ===Availability=== Program is actually available in the following versions: * '''''Molden 5.1''''' -- module <code>molden-5.1</code> (or just <code>molden</code>) * ''Molden 4.8'' -- module <code>molden-4.8</code> * ''Molden 4.7'' -- module <code>molden-4.7</code> ===Use=== Load the module (<code>module add molden</code>) and run <code>molden</code> (or other commands -- <code>ambfor</code>, <code>ambmd</code>, <code>gmolden</code>, <code>molden</code> or <code>surf</code>). ===Documentation=== [http://www.cmbi.ru.nl/molden/molden.html On-line] documentation is available. ===Supported platforms=== SGI Irix, Linux, FreeBSD, Mac OS, HP, Solaris, Alpha, Sun OS, Cray, OS2, VMS/OpenVMS, MS Windows ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://www.cmbi.ru.nl/molden/molden.html [[Kategorie:Applications]] [[Kategorie:Applications with GUI]] 6rtfzk7ysrzepg2z9f7l1itidr2h0ru 0 159 5062 5053 2016-05-13T09:13:18Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--ACT|Česká verze--> ===Description=== ACT is a free tool for displaying pairwise comparisons between two or more DNA sequences. It can be used to identify and analyse regions of similarity and difference between genomes and to explore conservation of synteny, in the context of the entire sequences and their annotation. ===Availability=== Program is available in version 7 ===Use=== Initialize enviroment ("module add act-7") and run "act". ===License=== GNU GPL. ===Documentation=== Documentation is available on [http://www.sanger.ac.uk/resources/software/act/ http://www.sanger.ac.uk/resources/software/act/]. ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://www.sanger.ac.uk/resources/software/act/ [[Kategorie:Applications]] 7mbnzb3ryyzafv5z4jfbbo1v4kvnyeu 0 160 5133 4736 2016-05-13T09:14:22Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Artemis|Česká verze--> ===Description=== Artemis is a free genome browser and annotation tool that allows visualisation of sequence features, next generation data and the results of analyses within the context of the sequence, and also its six-frame translation. ===Availability=== Program is available in version 10 ===Use=== Initialize enviroment ("module add art-10") and run "art". ===License=== GNU GPL. ===Documentation=== Documentatio n is available on [http://www.sanger.ac.uk/resources/software/artemis/ http://www.sanger.ac.uk/resources/software/artemis/]. ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://www.sanger.ac.uk/resources/software/artemis/ [[Kategorie:Applications]] d9ttwt7v12kflla98vopgpfv7d33zw6 0 163 5050 4991 2016-05-13T09:13:08Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--LDDist|Česká verze--> ===Description=== LDDist is a Perl module implemente in C++ that enables the user to calculate LogDet pair-wise genetic distances based on amino acid as well as DNA/RNA sequences. Site-to-site rate heterogeneity is handled by excluding invariant sites and/or bu assigning sites to different rate classes that are treated separately. ===Availability=== Program is available in version 1.3.2 ===Use=== Initialize enviroment ("module add lddist-1.3.2") and run "PLD.pl". ===License=== PAL (Perl Artistic License) ===Documentation=== Documentation is available on [http://artedi.ebc.uu.se/molev/software/LDDist.html WWW server]. ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://artedi.ebc.uu.se/molev/software/LDDist.html [[Kategorie:Applications]] 48ygerdfzlym5p8cwa3mlquv3vge9df 0 164 4956 4919 2016-05-13T09:11:21Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Migrate|Česká verze--> ===Description=== Migrate estimates effective population sizes and past migration rates between n population assuming a migration matrix model with asymmetric migration rates and different subpopulation sizes. Migrate uses maximum likelihood or Bayesian inference to jointly estimate all parameters. It can use the following data types: *Sequence data using Felsenstein's 84 model with or without site rate variation, *Single nucleotide polymorphism data (sequence-like data input, HAPMAP-like data input) *Microsatellite data using a stepwise mutation model or a brownian motion mutation model (using the repeatlength input format or the fragment-length input format), *Electrophoretic data using an 'infinite' allele model. The output can contain: Estimates of all migration rates and all population sizes, assuming constant mutation rates among loci or a gamma distributed mutation rate among loci. Profile likelihood tables, Percentiles, Likelihood-ratio tests, and simple plots of the log-likelihood surfaces for all populations and all loci. ===Availability=== Program is available in version 3.1.6, 3.2.20 and '''3.6.10'''. ===Use=== Inicialize environment by command module add migrate 3.6.10 then you can run single core, thread or MPI version of migrate by commands migrate-n-single migrate-n mpirun migrate-n-mpi For older versions: Initialize environment for one processor "module add migrate-3.1.6" or "module add migrate-3.2.20", for multi-processor MPI version "module add mpich-p4 migrate-3.1.6-mpi" or "module add openmpi migrate-3.2.20-mpi" for SHM version "module add migrate-3.1.6-thread" or "module add migrate-3.2.20-thread" and run "migrate-n". ===Documentation=== Documentation is available on [http://popgen.sc.fsu.edu/Migrate/Migrate-n.html WWW server]. ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://popgen.sc.fsu.edu/Migrate/Migrate-n.html [[Kategorie:Applications]] er0xf9vuomooengvon23rac5n7kilv0 0 167 4932 4885 2016-05-13T09:10:57Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--PhyML|Česká verze--> ===Description=== PhyML is a software that estimates maximum likelihood phylogenies from alignments of nucleotide or amino acid sequences. It provides a wide range of options that were designed to facilitate standard phylogenetic analyses. The main strengths of PhyML lies in the large number of substitution models coupled to various options to search the space of phylogenetic tree topologies, going from very fast and efficient methods to slower but generally more accurate approaches. It also implements two methods to evaluate branch supports in a sound statistical framework (the non-parametric bootstrap and the approximate likelihood ratio test). ===Availability=== Program is available in version 3.0 ===Use=== Initialize enviroment for * one processor run module add phyml-3.0 phyml * multi processor run module add phyml-3.0-mpi mpiexec phyml our you can use version with Intel compiler module add phyml-3.0-impi mpirun phyml-mpi -i InputFile ===License=== GNU GPL. ===Documentation=== Documentation is available on [http://code.google.com/p/phyml/ WWW server]. ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://code.google.com/p/phyml/ [[Kategorie:Applications]] 4alza4ktn717lkaxaxk7u685deh8eyc 0 168 5208 5207 2016-05-13T09:15:32Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--QUEEN|Česká verze--> ===Description=== Nuclear Overhauser effect (NOE) data are an indispensable source of structural information in biomolecular structure determination by NMR spectroscopy. Program QUEEN serves for quantitative evaluation of experimental NMR restrains. Program QUEEN for proper running needs program [[X-PLOR_application|X-Plor]]. ===Availability=== Version 1.1 ===Use=== Load module queen ("module ad queen") and run queen (in case you are using CHARON system, you have to use command "metamodule" to load this program). some solved questions are available on the FAQs. Way of loading a configuration In original program file queen.conf is read from the directory with program, in installed version you are allowed to set a path to you own configuration file into environment variable QUEEN_CONF. In case that standard configuration comply with all you needs except of settings of project directory, you can set this directory into environment variable Q_PROJECT and make no changes into QUEEN_CONF. For program efective running you have to put subdirectory "log" in assigment (item Q_LOG in queen.conf) into the fastest storage -- e.g. on a scratch volume (in a directory dedicated for a job -- identified by the <code>$SCRATCHDIR</code> environment variable) or in a /storage volume. There are available a standard program queen.py and new created script "queen", which after setting the same parametres as in the one processor job, make initialization of MPI, parallel computing on all processor which was mentioned in the assigment of the job and finally cancel MPI environment (cancel assistant processes) Suppored platforms QUEEN is availaible (and runs in primary mode) for all platforms in MetaCentrum exept of SGI (so in: 386, x86_64 a ia64 (Itanium)) and runs also for paralell running (through more processors or more nodes). ===Documentation=== http://www.cmbi.ru.nl/software/queen/ ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.cmbi.ru.nl/software/queen/ ===FAQs=== '''Q''':I can not run queen properly at testing example (see /home/pavelm/queen/test) In case I run queen locally in directory /home/pavelm/queen/test (by command "queen --Iuni example noe") the job runs properly, in case I want to queue the job, it fails with error message: "ERROR - Dictionary /home/pavelm/queen/test/queen.conf could not be read. Script has been stopped." I tryied to run this script by psubmit and also by qsub in normal queue ("qsub -q normal run" nebo "psubmit normal run"). Script contains both "metamodule add queen" and setting of path to configuration file into environment variable QUEEN_CONF ("QUEEN_CONF=/home/pavelm/queen/test/queen.conf;export QUEEN_CONF"). Script to execute is following: metamodule add queen Q_PROJECT=/home/pavelm/queen/test/ export Q_PROJECT QUEEN_CONF=/home/pavelm/queen/test/queen.conf export QUEEN_CONF #export Q_PROJECT=/home/pavelm/queen/test/ #export QUEEN_CONF=/home/pavelm/queen/test/queen.conf queen --Iuni example noe Unfortunatelly I can not learn why my configuration file can not be read. '''A''':Path which is set into QUEEN_CONF must be available from machine, where PBS ran your job (in case of parallel jobs from all nodes), so path /home/pavelm/... is local for clusters (leads on different clusters to different disks) Either put queen.conf into your directory in AFS(/afs/ics.muni.cz/home/pavelm, or /home/pavelm/shared) this directory is seen from all MetaCentrum machines, or limit running of job just to machines, which have access to your directory. E.g: by option PBS -l brno (-l ...:brno) if you have it in /home on skirit or perian. ...For program efective running you have to put subdirectory "log" in assigment (item Q_LOG in queen.conf) into the fastest storage -- e.g. on a scratch volume (in a directory dedicated for a job -- identified by the <code>$SCRATCHDIR</code> environment variable) or in a /storage volume. '''Q''':I linked through Q_LOG on $SCRATCHDIR ... I want to ask, if I can apply above described use of $SCRATCHDIR to parallel job in program xplor-nih. '''A''':According documentation (/afs/ics.muni.cz/software/xplor-nih-2.20/parallel.txt) you have to have a directory shared by all used nodes using program xplor in parallel mode. The best way should be to use the shared scratch volume (available just at the mandos cluster -- the dedicated space is identified by the <code>$SCRATCHDIR</code> environment variable), or to use a NFSv4 volume within the /storage directory. By the way, QUEEN must be able to read assingment from all nodes (e.g. it could be copied for just for reading), but subdirectory log could and it is sufficient to be on each node local, because some parts of a job run in it as separely runned xplor instance which communicate with queen local processes through files. Result of whole job is written just by main process. [[Kategorie:Applications]] qbe8n60hi7ft5iq93uar63ww0czv3ew 0 169 4856 4827 2016-05-13T09:09:20Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--X-PLOR|Česká verze--> ===Description=== X-PLOR is a program system for computational structural biology. X-PLOR's main focus is the three-dimensional structure determination of macromolecules using crystallographic diffraction or nuclear magnetic resonance (NMR) data. The program is based on an energy function approach: arbitrary combinations of empirical, geometric and effective energy terms describing experimental data may be used. The combined energy function can be minimized by a variety of gradient descent, simulated annealing, and conformational search procedures. ===Use=== Load module "xplor-nih" (module ad xplor-nih) and run "xplor-nih". ===Documentation=== http://nmr.cit.nih.gov/xplor-nih/doc/current/ ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://nmr.cit.nih.gov/xplor-nih/ [[Kategorie:Applications]] n74wj7jb3hgm5qg13dmfh69h22vnbdf 0 170 5147 5049 2016-05-13T09:14:34Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--TNT|Česká verze--> ===Description=== TNT stands for "Tree analysis using New Technology". It is a program for phylogenetic analysis under parsimony (with very fast tree-searching algorithms), as well as extensive tree handling and diagnosis capabilities. ===Availability=== Program is available in version 1.1 ===Use=== Initialize enviroment ("module add tnt-1.1") and run "tnt". ===License=== GNU GPL. ===Documentation=== Documentation is available on [http://tnt.insectmuseum.org/index.php/Main_Page WWW server]. ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://www.zmuc.dk/public/phylogeny/tnt/ [[Kategorie:Applications]] sfa7dqssmgc3de7jhl9518ptfsldub0 0 172 5116 4883 2016-05-13T09:14:07Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Maple|Česká verze--> = Description = Maple is a general-purpose computer algebra system -- an enviroment for sympolic computations, solving scientific and engeneering problems, mathematic research, and/or data visualization, thus allowing a creation of technical publications. = Usage = The following usage examples are '''specialized on and tested with the newest Maple version available (Maple 16)'''; their application on the older versions is possible, but might not work. Once submitting a job, which is intended to perform Maple computations, it is necessary to provide the scheduler with a '''requirement for available Maple license''' (thus, the job will be started just once a license becomes available). This requirement is specified by setting the '''maple''' job's property, as shown on the following examples (for another license reservation options see the [[#License reservation]] section below). ==== Interactive use ==== * Provide the scheduling system with a requirement for the number of available nodes/cores and Maple licenses: <source lang=bash> qsub -I -l nodes=1:ppn=X:... -l maple=1 # requires 1 license, 1 node having X cores </source> * Initialize the Maple environment ("''module add maple''") and run the command: ** ''maple'' ... for interactive text environment ** ''xmaple'' ... for interactive graphical environment *** if the interactive graphical environment is required, it is necessary to have a connection to an X-server, see the page [[X-Window]] for details ==== Batch use ==== Create a job startup script, which initially performs an intialization of the modules subsystem, followed by an initialization of the Maple environment. After that, the script might perform the Maple computations intended: <source lang=bash> #!/bin/sh # initialize the Maple module add maple # perform Maple computations maple < mymaplefile.{txt,mpl} </source> Subsequently, submit the script together with your requirements for the number of available nodes/cores and Maple licenses: <source lang=bash> qsub -l nodes=1:ppn=X:... -l maple=1 mujskript.sh # requires 1 license, 1 node having X cores </source> === Integrating Maple with Matlab === The installed '''Maple Toolbox for Matlab''' allows two-way integration between Maple and Matlab environments -- the toolbox thus allows to combine the Maple's symbolic computations with Matlab's numeric computations. See more information about the toolbox at [http://www.maplesoft.com/products/maple/features/feature_detail.aspx?fid=6721 Maple official site]. '''Usage example (interactive job):''' * ask the scheduling system for an interactive job <source lang=bash> qsub -I -l nodes=1:ppn=X:... -l maple=1 # X ... the number of required cores </source> * initialize the ''matlab'' module <source lang=bash> module add matlab </source> * start the Matlab having the Maple symbolic toolbox enabled <source lang=bash> matlab-sym-maple </source> * write the following set of commands into the Matlab worksheet: <source lang=bash> maple % starts the Maple (might take some time) syms x y % initializes symbolic variables cos(sqrt(x^2-y^2))/x^2 setmaple('h',ans) % allows the result to be accessible from the Maple </source> * an access to the resulting formula ('''h''' variable) from the Maple environment can be checked by typing the '''h''' into the Maple worksheet and pressing Enter -- the Matlab formula should be shown. For more detail (e.g., a batch toolbox usage), please visit the '''[[Matlab]] section''' of this Wiki or visit the '''[http://www.maplesoft.com/products/maple/features/feature_detail.aspx?fid=6721 Maple official pages]'''. A '''very useful videopresentation''' illustrating the toolbox usage together with its capabilities can be found [http://www.maplesoft.com/demo/streaming/symboliccomputingusingmtm.aspx here]. = Documentation = * The documentation is available at the [http://www.maplesoft.com/documentation_center/ Maple official site] or locally in the ''Maple directory''. = Licenses = * ''Maple 16:'' '''30''' floating liceses * ''Maple 15:'' '''30''' floating liceses * ''Maple 11:'' '''1''' floating liceses * ''Maple 10:'' '''1''' floating liceses * ''Maple 5:'' '''unlimited number of licenses''', but limited to the use from the Supercomputing Centre Brno nodes {{Warning | The purchased licenses permit '''just an academic use''' of the program!}} = Supported platforms = Linux, SGI IRIX (various versions) = Program administrator = Tomáš Rebok (rebok at ics.muni.cz) = Homepage = URL: http://www.maplesoft.com/products/maple/index.aspx = License reservations = As mentioned above, the Maple application is available in different versions: * when using a '''floating license''' (Maple 10, 11, 15 and 16), it is necessary to ask the scheduling system for a license reservation via the job submission attributes. Currently, there are for types of Maple licenses available via the scheduling system attributes: ** maple Maple 16, 15 ** maple11 Maple 11 ** maple10 Maple 10 ** maple10p Maple 10+ i.e., whichever is free * since the Maple 5 has its '''license limited to a location''', it is necessary to start the Maple from the appropriate nodes ====Maple 16==== The Maple 16 is able to use just more computational threads (i.e., cores on a single node). It is necessary to ask for a floating license. Thus, the job submission (requiring a single computational node) should be performed as follows: <source lang="bash"> qsub -l nodes=1:ppn=4:x86_64,mem=16gb,maple=1 ... </source> i.e., the job is asking for four cores on a single 64-bit node, 16GB of memory and a single Maple 16 license. ====Maple 15==== The Maple 15 is able to use both more computational threads (i.e., cores on a single node) and more distributed nodes (once the Grid toolbox becomes availabe -- see above). It is necessary to ask for a floating license. Thus, the job submission (requiring a single computational node) should be performed as follows: <source lang="bash"> qsub -l nodes=1:ppn=4:x86_64,mem=16gb,maple=1 ... </source> i.e., the job is asking for four cores on a single 64-bit node, 16GB of memory and a single Maple 15 license. ====Maple 11==== Maple 11 is able to use more computing threads (cores) on a single node. The list of parallelized internal functions is limited (primarily matrix functions with floats), however, it is possible to create own parallelized applications. It is necessary to ask for a floating license. The job submission will thus look like as follows: <source lang="bash"> qsub -l nodes=1:ppn=4:x86_64,mem=16gb,maple11=1 ... </source> i.e., the job is asking for four cores on a single 64-bit node, 16GB of memory and a single Maple 11 license. ====Maple 10==== The purchased Maple 10 is able to use just a single processor (on a single node). It is necessary to ask for a floating license. The job submission will thus look like as follows: <source lang="bash"> qsub -l nodes=1:ppn=1:x86_64,mem=16gb,maple10=1 ... </source> i.e., the job is asking for a single core on a single 64-bit node, 16GB of memory and a single Maple 10 license. ====Maple 10 or 11==== If it's not important, which version (10 or 11) will perform the computation, it is possible to ask for a "maple10p" license, which will result in reserving any of the available ones (for Maple 10 or 11). In such a case, the "maple" command has to be used for invoking Maple -- the command will start Maple 10, if a license for Maple 10 is available, or Maple 11, if a license for Maple 11 is available. (The command line options are exactly the same as for the original "maple" command.). In such a case, the job submission will look like as follows: <source lang="bash"> qsub -l nodes=1:ppn=1:x86_64,mem=16gb,maple10p=1 tenplus.sh </source> and the file tenplus.sh looks as follows: <source lang="bash"> #!/bin/sh # initialize the maple-10 module module add maple-10 #run the maple maple10p <inputdata </source> ====Maple 5==== A historical version, primarily used because of its unlimited licenses (limited just to the use on Brno nodes). Once submitting a job, it is necessary to ask for a node located in Brno (there's no need to ask for a license reservation): <source lang="bash"> qsub -l nodes=1:ppn=1:brno,mem=16gb ... </source> [[Kategorie:Applications]] lui2p4ivha27rup9wmqcvw5o0mmr5gg 0 173 5223 5077 2016-05-13T09:19:46Z Root 1 wikitext text/x-wiki <!--([[Matlab|Česká verze]])--> =Description= MATLAB is an integrated system covering tools for symbolic and numeric computations, analyses and data visualizations, modeling and simulations of real processes etc. Next to MATLAB and Simulink there are other supplementary toolboxes available (see also section [[#Licenses|Licenses]] below). =Availability= On all machines one can load the currently available MATLAB version using [https://meta.cesnet.cz/wiki/Modules SW modules]. Currently it is '''MATLAB version 8.3''' (module <code>matlab</code>), however, the older versions -- 8.2, 8.1, 8.0 and 7.14 (modules <code>matlab-VER</code>, e.g. <code>matlab-8.0</code>) -- are also available... =Usage= Loading SW module matlab and running MATLAB: <source lang="bash"> $ module add matlab $ matlab </source> In the following text $MATLABROOT means the directory /software/matlab-8.3. Command line parameters are described [http://www.mathworks.com/help/techdoc/ref/matlabunix.html here]. {{Application with GUI}} ==== Starting the Matlab with Maple symbolic environment ==== Since the Maple's symbolic environment is [http://www.mapleprimes.com/posts/41280-Maple-Toolbox-For-Matlab-Is-NOT-Fully not fully compatible] with the Matlab's symbolic environment, the '''Matlab symbolic environment is used by default''' when starting the Matlab within MetaCentrum environment. To start Matlab having '''Maple symbolic environment enabled''', it is necessary to provide the system with an '''explicit requirement''' for it (using the following sequence of commands): <source lang="bash"> $ module add matlab $ matlab-sym-maple # the options of this command are the same as for the original 'matlab' command # an alternative -- the 'matlab-sym-matlab' command -- explicitly requires the Matlab symbolic environment </source> Let us refer you to the [http://www.maplesoft.com/demo/streaming/symboliccomputingusingmtm.aspx following video] for further information about using the Maple's symbolic environment within Matlab. =Documentation= <!-- Original paper documentation is available at administrators (ÚVT, office C122). --> From the MATLAB command window one can use the command help to get help with a a particular command, e.g. <source lang="matlab"> >> help rand </source> in the desktop environment one can use also the command doc <source lang="matlab"> >> doc rand </source> Documentation in PDF and HTML format is available in the directory $MATLABROOT/help. Online documentation is available at the address [http://www.mathworks.com/help/techdoc/matlab_product_page.html MATLAB Product Documentation]. =Licences= There exists a permanent licence type "College" (for operating systems UNIX and MS Windows) that is available to '''all the national grid infrastructure MetaCentrum users''' as well as to all students and employees of * Masaryk university in Brno, * ZČU in Plzeň and * ČVUT in Praha. MATLAB can be installed freely on every computer at these universities and once running it takes licences from a pool of [https://meta.cesnet.cz/osobni/personal/eula.do available licences]. These licences are currently maintained by three licence servers: at ZČU in Plzeň, at ÚVT UK in Praha and at ÚVT MU in Brno. {{Warning | The purchased licenses permit '''just an academic use''' of the program!}} Names and number of licences: *MATLAB 450 *SIMULINK 150 <table> <tr> <td> *Bioinformatics_Toolbox 15 *Communication_Toolbox 25 *Video_and_Image_Blockset 1 *Control_Toolbox 50 *Curve_Fitting_Toolbox 52 *Signal_Blocks 50 *Data_Acq_Toolbox 2 *Database_Toolbox 9 *Datafeed_Toolbox 1 *Econometrics_Toolbox 1 *RTW_Embedded_Coder 1 *Fin_Instruments_Toolbox 2 *Financial_Toolbox 2 *Fixed_Point_Toolbox 1 *Fuzzy_Toolbox 51 *GADS_Toolbox 1 *Image_Acquisition_Toolbox 2 *Image_Toolbox 31 *Instr_Control_Toolbox 1 *MATLAB_Builder_for_Java 5 *MATLAB_Coder 7 *Compiler 7 *Neural_Network_Toolbox 150 </td> <td> *Optimization_Toolbox 150 *Distrib_Computing_Toolbox 15 *PDE_Toolbox 50 *Real-Time_Win_Target 51 *Signal_Toolbox 50 *SimBiology 5 *SimHydraulics 1 *SimMechanics 4 *Power_System_Blocks 1 *Simscape 5 *Virtual_Reality_Toolbox 3 *Real-Time_Workshop 2 *Simulink_Control_Design 50 *Excel_Link 1 *Statistics_Toolbox 50 *Symbolic_Toolbox 150 *Identification_Toolbox 51 *Vehicle_Network_Toolbox 1 *Wavelet_Toolbox 8 *Simulink_HDL_Coder 1 *Embedded_IDE_Link 1 *Target_Support_Package 1 *MATLAB_Distrib_Comp_Engine 320 </td> </tr> </table> Together with the permanent licence a complete maintenance (including new version updates of all mentioned products) is also available and it is annually renewed. A list of licenses can be obtained by <source lang="bash"> $ /software/matlab-8.3/etc/lmstat -s skiritf.ics.muni.cz -a | grep "in use" </source> =Supported platforms= Current version supports the following operating systems: Solaris, HP-UX, Linux, MacOS-X and MS Windows, older versions are available also on: Digital Unix (Tru64), Irix and AIX. =Program administrator= [mailto:ruda@ics.muni.cz Miroslav Ruda ] =MATLAB Homepage= URL: http://www.matlab.com =Tips= * If asking for one CPU only, use matlab command line parameter '''-singleCompThread'''. * By using [https://meta.cesnet.cz/wiki/Spouštění_úloh_v_plánovači properties] '''matlab_Distrib_Computing_Toolbox=1''' and '''matlab_MATLAB_Distrib_Comp_Engine=1''' it is possible to prevent execution of the job if the licence is not available (the job remains waiting in the queue until the licence is available). See also below. * MATLAB uses its own java virtual machine for the desktop environment. It is possible to turn the JVM off (run matlab -nojvm) to speed up many jobs. WARNING: some internal functions and toolboxes (for example '''Ditributed Computing Toolbox''') need java. =Detailed HOWTO= In the following text we explain how to run an interactive/batch single/parallel MATLAB job in MetaCentre environment. ===Interactive job=== For an interactive work, Matlab can be run either on frontends or computing nodes (however, use the frontends just for a trivial non-demanding computations or data preparation/analysis). To get a computing node for an interactive work, as the scheduler for an [[interactive job]] at first (parameter <code>-I</code>). Once the interactive job starts, one may start and '''use Matlab both in text as well as graphical mode'''. To prepare the '''graphical environment''', follow the instructions available at [[remote desktop]] page. Then, the Matlab can be run from the provided menu or by typing the following code into a terminal window: <source lang="bash"> $ module add matlab $ matlab </source> <!-- <source lang="bash"> skirit$ qsub -I -q short -l nodes=1:ppn=1,mem=1000mb,matlab=1 </source> (a queue short is for jobs lasting less than 2 hours, nodes=1:ppn=1 means one CPU, mem=1000mb asks for 1000MB memory allocation, matlab=1 asks for one MATLAB licence) You will be redirected directly to the given machine. You can run MATLAB interactively, if you have an X-Windows server on your machine: <source lang="bash"> $ module add matlab $ export DISPLAY=my.machine.somewhere.cz:0 $ matlab ... $ exit </source> If you are behind a firewall or if your IP address is not public, you may need to tunnel the X-protocol through ssh (secure shell) using the parameter -X, i.e. on your machine with running X-server run <source lang="bash"> my_machine$ ssh -X machine_with_matlab.cz machine_with_matlab$ module add matlab; matlab </source> If you are using MS-Windows, you may need an X-server for MS-Windows, e.g. Xming, Cygwin/X, X-Win32, ReflectionX or Exceed. For ssh tunnelling of X-protocol you may use Putty or Cygwin.--> Alternatively, it is possible to run MATLAB just in the text regime only, when the graphical mode is not necessary (you may even create figures, work with them, save them to a disk, they are only invisible): <source lang="bash"> $ matlab -nosplash -nodisplay -nodesktop </source> Interactive regime does not bring any significant speed-up comparing to running MATLAB locally on your machine unless parallelism is used. Interactive regime may be used to development of your code, testing its execution (strongly recommended) and after the development you may run it in a batch. ===Batch job=== It is possible to run MATLAB without the desktop environment in the batch regime. Use arguments -nodisplay and -r. Create a shell script myjob1.sh with the following contents: <source lang="bash"> #!/bin/bash # set PATH to find MATLAB module add matlab # go to my working directory cd /storage/home/$LOGNAME/matlab/ # run MATLAB matlab -nosplash -nodesktop -nodisplay -r "myFunction()" # or in a differenet way: #matlab -nosplash -nodesktop -nodisplay < myFunction.m > output.txt </source> Put all your MATLAB files (your *.mat and *.m files) to the directory /storage/home/$LOGNAME/matlab/ that is shared on all machines. Submit this shell script by <source lang="bash"> qsub -q short -l nodes=1:ppn=1,mem=1000mb,matlab=1 myjob1.sh </source> Batch jobs are useful especially if you want to run more jobs in parallel or if you do not want to block your machine with running jobs. If you expect jobs to last more than two hours use the queue normal (up to 24 hours) or the queue long (up to 30 days). ===Distributed and parallel jobs in MATLAB=== MATLAB now supports distributed and parallel jobs on multiprocessor machines using the [http://www.mathworks.com/products/parallel-computing/ Parallel Computing Toolbox] (the name of the licence is Distrib_Computing_Toolbox) and on clusters using the [http://www.mathworks.com/products/distriben/ MATLAB Distributed Computing Server] (name of the licence is MATLAB_Distrib_Comp_Engine). :'''''Note:''' Since MATLAB version R2010b there is also an extended support for PBS and TORQUE schedulers that allows to easily run parallel/distributed jobs directly from the MATLAB environment -- see more at [[/Distributed computations using Torque|Distributed computations using Torque]].'' ====Parallel Matlab computations in MetaCentrum==== To prepare an environment for parallel computations, it is necessary to initialize a parallel pool of so-called ''workers'' using the function [http://www.mathworks.com/help/distcomp/parpool.html <code>parpool</code>] (called <code>matlabpool</code> in previous versions). This standard initialization requires to specify an amount of workers to initialize; moreover, thanks to shared filesystems, it may also result in a collision when trying to initialize several pools simultaneously. :<span style="color: darkred;">'''To make the initialization of parallel pool easier as well as to cope with the collision problems, we have prepared a function <code>MetaParPool</code>''':</span> <source lang=matlab> MetaParPool('open'); % initializes parallel pool (returns the number of initialized workers) ... x = MetaParPool('size'); % allows to discover the size of parallel pool (returns the number of workers) % may be called as MetaParPool('info'); as well ... % a computation using parfor, spmd and other Matlab functions ... MetaParPool('close'); % closing the parallel pool </source> :'''''Notes:''''' :* the function '''automatically detects the number of cores assigned to a job''' (max. 12) -- the size of parallel pool is always automatically set based on resources assigned to a job :** it is necessary to ask for N computing cores (max. 12) on a single node (<code>qsub -l nodes=1:ppn=N ...</code>) :* to make parallel computations using this function, there are '''1 Matlab license''' and '''1 Distributed Computing Toolbox license''' necessary :** a reservation can be thus performed via <code>-l matlab=1,matlab_Distrib_Computing_Toolbox=1</code> -- see [[#Tips]] :* '''an example of parallel computation using the <code>MetaParPool</code>''' function can be found in the <code>/software/matlab-meta_ext/examples</code> directory (file <code>example-parallel.m</code> shows the Matlab input file itself while the file <code>run_parallel.sh</code> shows an example startup script). :* more information about parallel computing in Matlab can be found [http://www.mathworks.com/products/parallel-computing/ on producer's webpage] ====Distributed Matlab computations in MetaCentrum==== Distributed computations in Matlab are commonly performed via parametrization of the internal scheduler (of type ''torque'') -- once a computation is ready, the internal scheduler starts new common PBS jobs, which perform the required functionality after their startup (see more information at [[/Distributed computations using Torque|Distributed computations using Torque]]). However, such behaviour may not be comfortable -- the master process (preparing the distributed computation) may wait for an undefined time to finish the created PBS jobs; also, the parametrization of the internal scheduler as well as nodes' specification may also not be convenient. :<span style="color: darkred;">'''To make the preparation of the distributed pool easier, we have prepared a function <code>MetaGridPool</code>''':</span> <source lang=matlab> js = MetaGridPool('open'); % initializes distributed pool (returns the internal scheduler for preparation of the computation) ... x = MetaGridPool('size'); % returns the size of the initialized pool (number of workers) MetaGridPool('info'); % shows detailed information about the initialized pool ... % distributed computation (functions createJob, createTask, wait, etc.) ... MetaGridPool('close'); % closes the distributed pool </source> :'''''Notes:''''' :* the function '''automatically detects the number of nodes and cores assigned to a job''' -- the size of the pool is always set based on resources really reserved for a job (''the number of workers is decreased by 1 -- a single core is reserved for the master process'') :** it is necessary to ask for X computing nodes and Y computing cores (<code>qsub -l nodes=X:ppn=Y ...</code>) :* to perform distributed computations, there are '''1 Matlab license''', '''1 Distributed Computing Toolbox license''' and '''N-1 MATLAB Distributed Computing Engine licenses''' necessary ('''N''' denotes the number of requested cores) :** a reservation can be thus performed via <code>-l matlab=1,matlab_Distrib_Computing_Toolbox=1,matlab_MATLAB_Distrib_Comp_Engine=7</code> -- see more at [[#Tips]] :* '''an example of distributed computation using the <code>MetaGridPool</code>''' function can be found in the <code>/software/matlab-meta_ext/examples</code> directory (file <code>example-distributed.m</code> shows the Matlab input file itself while the file <code>run_distributed.sh</code> shows an example startup script). :* more information about distributed computing in Matlab can be found [http://www.mathworks.com/products/distriben/ on producer's webpage] =Useful links= *[http://www.mathworks.com/products/parallel-computing/demos.html Parallel Computing Toolbox Demos] *[http://www.mathworks.com/products/distriben/demos.html?file=/products/parallel-computing/includes/parallel.html Parallel Programming in MATLAB] *[http://www.mathworks.com/products/parallel-computing/demos.html?file=/products/demos/shipping/distcomp/paralleltutorial_dividing_tasks.html Dividing MATLAB Computations into Tasks] *[http://www.mathworks.com/products/parallel-computing/demos.html?file=/products/demos/shipping/distcomp/paralleltutorial_taskfunctions.html Writing Task Functions] *[http://www.mathworks.com/products/parallel-computing/demos.html?file=/products/demos/shipping/distcomp/paralleltutorial_network_traffic.html Minimizing Network Traffic] [[Category:Applications]] [[Category:English]] 6ja3ev7ralnwhywafe47d7c72fm4uo0 0 175 5057 4851 2016-05-13T09:13:13Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Mathematica|Česká verze--> =Description= '''Mathematica''' is next CAS (Computer Algebra System) like Matlab and Maple. It is able to analyze data sets, solve complex differential equations, and develop an entire solution for fast, high-precision numeric and symbolic computation. Mathematica also provides 2D and 3D data visualization and programming capabilities. {{Rámeček | text='''''Note:''''' To enable '''computational-intensive computations''' within Mathematica, one can make use of its '''parallel/distributed extension''' called '''GridMathematica''' -- see details at [[GridMathematica application|GridMathematica documentation]].}} =Availability= * '''module <code>mathematica-10</code>''' -- Mathematica version 10.0.1, '''10 concurrent runs''', the license purchased by the CERIT-SC Centre, '''available for all the registered MetaVO users'''. * ''module <code>mathematica-9</code>'' -- Mathematica version 9.0.1, ''10 concurrent runs'', the license purchased by the CERIT-SC Centre, ''available for all the registered MetaVO users''. * '''modules <code>mathematica-9-XXX</code>''' -- Mathematica version 9.0.1, installations dedicated for use by the institutions owning a Mathematica license (actually, ''XXX = JCU, FZU or UK''). * ''modules <code>mathematica-8-XXX</code>'' -- Mathematica version 8, installations dedicated for use by the institutions owning a Mathematica license (actually, ''XXX = JCU, FZU or UK''). :'''''Note:''' If you are interested in using your Mathematica license within the MetaCentrum infrastructure, contact us for more details (the license terms and conditions of your license have to be consulted if such a use is available).'' =License= Commercial software, '''just for academic use'''. {{Warning | The purchased licenses permit '''just an academic use''' of the program!}} =Use= Initialize enviroment ("<code>module add mathematica-9</code>" (version 9) or (in cases your institution has Mathematica licenses) "<code>module add mathematica-9-XXX</code>" or "<code>module add mathematica-8-XXX</code>") and run "Mathematica": <source lang="bash"> $ Mathematica # GUI version $ math # command-line version </source> '''''Note:''' To enable computational-intensive computations within Mathematica, one can make use of its parallel/distributed extension called '''GridMathematica''' -- see details at [[GridMathematica application|Gridmathematica documentation]].'' {{Application with GUI}} =Documentation= * http://www.wolfram.com/mathematica/ * Mathematica and Maple comparison: http://www.wolfram.com/mathematica/compare-mathematica/compare-mathematica-and-maple.html =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://www.wolfram.com/mathematica/ [[Kategorie:Applications]] [[Category:English]] teyaze8ygfzdenjq8dstcyxaqo4088d 0 176 4896 4804 2016-05-13T09:10:08Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--MSC.Marc|Česká verze--> ===Description=== MSC.MARC made by MSC.Software company is program system based on finite element method and serves for solving large linear and non linear (physical and geometrically), structural, thermal, thermal-mechanical, electromagmetic and acoustic jobs. Program is highly appreciated for application in impulse phenomenon area and large deformation problem. ===Availability=== Modul marc2005 for version MSC.MARC 2005. ===Use=== MSC.MARC can be run in MetaCentrum enviroment after connecting particular module by command "marc2005".Computing enviroment of MetaCentrum serves primary for proceeding sophisticated computings (preprocessing and postprocesing can run in the user workstation). Several use cases with MSC.MARC ilustrate commneted scripts for PBS.(Section is under construction. ===Documentation=== Electronic program documentation is its integred part. Another manuals are available on the ZCS page in Plzeň. ===Licence=== Product is licenced for students and employees of ZČU in Plzeň. There are pool 6 flowing licences available and another 24 licences for use another processors during parallel computings. fluent 30 licences most, gambit 10 licences most, trigrid 30 licences most, fidap 2 licences most. Number of licences, which are temporary available for use to run PBS job you can find with typping command "qfree -i". ===Maintenance=== Annual rent of software is payed from resources of ZČU in Plzeň. ===Supported platforms=== All UNIXové platforms and MS Windows, there is MSC.Marc in METACentrum installed for platform Linux_ia32. ===Program administrator=== [mailto:knourek@civ.zcu.cz Jindřich Kňourek] ===Homepage=== URL:http://www.mscsoftware.com [[Kategorie:Applications]] nmkw9vgsug1x0tylxomm12cq3ei5act 0 178 5174 4922 2016-05-13T09:15:00Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--WIEN2k|Česká verze--> ===Description=== Program WIEN2k computes the electronic structure of solids. It is based on the Density Functional Theory (DFT) [W. Kohn and L. J. Sham: Phys. Rev. 140, A1133–A1138 (1965)]. It comprises full crystal potential and uses the Linearized Augmented Plane Wave method, LAPW [S. Cottenier: DFT and the Family of (L)APW-methods: a step-by-step introduction, University of Ghent, Belgium, 2004]). ===Availability=== Program is actually available in the following versions: * '''WIEN2K-14.2''' -- module <code>wien2k-14.2</code> (supports distributed/MPI computations) works with no problems. * WIEN2K-14.1 -- module <code>wien2k-14.1</code> (supports distributed/MPI computations) works with no problems. * WIEN2K-13.1 -- module <code>wien2k-13.1</code> (supports distributed/MPI computations) works with no problems. The recommendation is to use wien2k-08 for w2web and wien2k-13.1 for computing. * WIEN2K-12.1 -- module <code>wien2k-12.1</code> (supports distributed/MPI computations) works with no problems. The recommendation is to use wien2k-08 for w2web and wien2k-12.1 for computing. * WIEN2K-10.1 -- module <code>wien2k-10</code> * WIEN2K-08 -- module <code>wien2k-08</code> works with no problems * WIEN2K-05 -- module <code>wien2k-05</code> ''Note:'' Since version wien2k-12.1 does not yet implement MPI parallelization. However, it uses the parallelization of Open MP. Your task will use so many processors how many you specify in the qsub command. For example, for two-processor task you submit: <source lang="bash"> $ qsub -q normal -l nodes=1:ppn=2:nfs4,mem=1gb,vmem=1gb .... </source> '''''Do not use the -p switch''''' when using run_lapw. Just <source lang="bash"> export SCRATCH=. run_lapw -cc 0.0001 -i 90 echo y | clean_lapw -s </source> is enough for computing on more processors. ===Licence=== Wien2k is a commercial product -- every user '''should have it's own (private) license''' (see [http://www.wien2k.at/order/cc-files.html license terms and conditions]). '''''Note:''' Including the version 14.1, the program directories are accessible only for the users which sent us a proof of holding the license...'' ===Use=== Load the module of the desired version (e.g., "module add wien2k-12.1"), which makes all the Wien2k binary files available for use. ===Documentation=== http://www.wien2k.at/ ===FAQs=== '''Q''': There are processes left on machines after calculations. '''A''': Depends which mode of calculation you use. * for interactive tjob you just simply look if there are some processes left on machines after finishing your task. These then can be canceled using: <source lang="bash"> $ ps -u YOURLOGIN $ echo y | clean_lapw $ echo y | cancel_lapw </source> * for batch job, you should add to your running script this line: <source lang="bash"> trap 'echo y | clean_lapw; echo y | cancel_lapw ' SIGTERM </source> '''Q''': How to use WIEN2k only in text regime only (no graphics)? '''A''': It is possible. The problem is, that the w2web interface does not work on your local computer. In fact, you must launch a graphic browser ''iceweasel'' directly on the MetaCentrum machine. You may temporarily try to use non-graphic browser ''lynx'' or its later version ''links''. This works. <source lang="bash"> $ lynx </source> After pressing the key '''g''' in lynx, you may specify the adress: <source lang="bash"> http://nympha.zcu.cz:7890 </source> There, you can specify your struct file, run a calculation and so on. There is another way to specify your structure file, like this: <source lang="bash"> $ cp $WIENROOT/SRC_templates/case.struct ./myStructFile.struct </source> Some examples of structure files can be found in $WIENROOT/example_struct_files '''''Beware''', the structure file is formatted. After editing structure file you may initialize your structure in text regime using <source lang="bash"> $ init_lapw </source> '''Q''': How to make calculations using WIEN2k if one has graphic support. '''A''': Log in for examle on nympha.zcu.cz. <source lang="bash"> $ ssh -X myUserName@nympha.zcu.cz </source> In your home directory, create the directory TiC and launch Wien2k. For example (change manik to your user name): <source lang="bash"> $ pwd /storage/plzen1/home/manik $ mkdir TiC $ cd TiC $ module add wien2k $ w2web </source> Running first w2web: The computer asks you your user name and password. Choose non-trivial password ('''different''' from your MetaCentrum password). If there is an announcement: <source lang="bash"> w2web failed to bind port 7890 - port already in use! You may want to try w2web -p portnumber </source> Then you should erase the w2web profile and try another port: <source lang="bash"> $ rm -r ~/.w2web $ w2web -p 7891 </source> If this happens, very probably some other user has forgotten to kill the process of his w2web. We kindly ask you not to forget to kill the process of w2web after finishing your work: <source lang="bash"> $ ps -u manik PID TTY TIME CMD 19082 pts/0 00:00:00 bash 19141 ? 00:00:00 w2web 19142 pts/0 00:00:00 ps $ kill 19141 </source> Then launch the browser ''iceweasel'' on (for example) nympha.zcu.cz <source lang="bash"> $ iceweasel & </source> In the address field, type http://nympha.zcu.cz:7890 (eventually higher number of port 7891 or more, if 7890 is in use). After typing your user name and password, you follow-up the [http://www.wien2k.at/reg_user/textbooks/usersguide.pdf WIEN2k Usersguide]. In case of problems you can search WIEN2k [http://www.wien2k.at/reg_user/faq/ WIEN2k FAQ] or search the [http://www.wien2k.at/reg_user/mailing_list/ WIEN2k Mailing List]. As a last resort, you can join the mailing list and pose your own question there. (http://www.wien2k.at/reg_user/index.html). You can use the [[XCrysDen]] code to visualize your structure files and other task in WIEN2k. <source lang="bash"> $ cd TiC $ module add xcrysden $ xcrysden & </source> and choose File / Open WIEN2k / Open WIEN2k Struct File / TiC.struct. If you are using w2web, you may view the structure directly by clicking on ''view structure'' in ''Utils.'' '''Q''': sgroup crashes when initializing my structure '''A''': This was the problem in old version wien2k-05. Use higher version wien2k-12.1 or at least wien2k-08: <source lang="bash"> $ module rm wien2k-05 $ module add wien2k-12.1 </source> '''Q''': First iteration crashes (after lapw0) and there is error announcement in file case.dayfile: ''Segmentation fault'' '''A''': Before doing ''run_lapw'' or ''runsp_lapw'' type to command line (or add to your running script): <source lang="bash"> $ ulimit -s unlimited </source> This works for default ''bash'' shell. If using ''csh'' or ''tcsh'', type <source lang="bash"> limit stacksize unlimited </source> (The following command tells your which shell you are using) <source lang="bash"> echo $SHELL </source> '''Q''': I get the message: L2main - QTL-B Error '''A''': Have a look at [http://www.wien2k.at/reg_user/faq/qtlb.html WIEN2k FAQ] page concerning this. '''Q''': How do I know which version of WIEN2k I am currently using: '''A''': By: <source lang="bash"> $ module list </source> or <source lang="bash"> $ which run_lapw </source> Safe mode to switch the WIEN2k versions is: <source lang="bash"> $ module rm wien2k-08 $ module add wien2k-12.1 </source> '''Q''': I want to make an immediate calculation now. '''A''': Firstly, '''it is forbidden to make calculation directly on frontends'''. ''Simply use the interactive job.'' <source lang="bash"> $ qsub -q normal -l nodes=1:ppn=1:nfs4,mem=1gb -I qsub: waiting for job 135447.arien.ics.muni.cz to start </source> wait a minute ... <source lang="bash"> qsub: job 135447.arien.ics.muni.cz ready </source> and you are on (for example) node skirit28. <source lang="bash"> $ hostname skirit28.ics.muni.cz $ pwd /storage/brno1/home/manik/ $ cd /storage/plzen1/home/manik/TiC $ module add wien2k-12.1 $ ulimit -s unlimited $ export SCRATCH=. $ run_lapw -cc 0.000001 -i 90 </source> Wait till your calculation finishes and then tidy up your job: <source lang="bash"> $ clean_lapw $ exit qsub: job 135452.arien.ics.muni.cz completed </source> '''Q''': Is there any default running script for WIEN2k? '''A''': Yes. When submitting this script, you must be in the ''case'' directory of your calculation. <source lang="bash"> #!/bin/bash #PBS -j oe module add wien2k-12.1 ulimit -s unlimited hostname echo "Your directory is $PBS_O_WORKDIR" cd $PBS_O_WORKDIR export SCRATCH=. run_lapw -cc 0.0001 -i 90 echo y | clean_lapw > /dev/null hostname </source> ===Supported platforms=== Linux ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://www.wien2k.at/ [[Kategorie:Applications]] [[Kategorie:English]] 4vmeewpft58f9akq5mvyp9x1knh7j32 0 183 5196 4965 2016-05-13T09:15:17Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Scipy|Česká verze--> ===Description=== Scipy is another language extension that uses numpy to do advanced math, signal processing, optimization, statistics and much more. ===Availability=== * '''Scipy version 0.12.0:''' compiled with Intel MKL 11.0 '''(internally parallelized)''', which results in '''vastly improved computation performance''' ** module <code>scipy-0.12.0-py2.7</code> *** prepared to be used with Python 2.7 (module ''*-py2.7'') * ''Scipy version 0.11.0:'' compiled with Intel MKL 11.0 '''(internally parallelized)''', which results in '''vastly improved computation performance''' ** modules <code>scipy-0.11.0-py2.5</code> and <code>scipy-0.11.0-py2.6</code> *** prepared to be used both with Python 2.5 (module ''*-py2.5'') as well as Python 2.6 (module ''*-py2.6'') * ''Scipy version 0.10.1:'' compiled with standard BLAS libraries ** modules <code>scipy-0.10.1-py2.5</code> and <code>scipy-0.10.1-py2.6</code> *** prepared to be used both with Python 2.5 (module ''*-py2.5'') as well as Python 2.6 (module ''*-py2.6'') ===Use=== Load module (<code>module ad "module_name"</code>), then you are allowed to import ''scipy'' libraries into Python. '''''Note:''' The modules version 0.10.1 do not automatically load the Python modules -- if you need them, load them manually.'' ===Documentation=== [http://docs.scipy.org/doc/ On-line] documentation is available. ===Supported platforms=== Linux ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://www.scipy.org/Getting_Started [[Kategorie:Applications]] bnv4e7f3gww2xopi7hir83x3n3wmkqc 0 186 4979 4911 2016-05-13T09:11:51Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--PGI_CDK|Česká verze--> =Description= PGI Cluster Development Kit is a collection of tools for development parallel ( supports parallel models OpenMP a MPI) also seriál programs in laguanges C, C++, FORTRAN 77, Fortran 95 and High Performance Fortran (HPF). Program exept to compilers and supported libraries contains debugger and profiler with support for parallel programs, optimalized mathematic libraries BLAS and LAPACK and parallel library ScaLAPACK. =Availability= * PGICDK version '''14.9''' (module <code>pgicdk-14.9</code>): '''''2 permanent network-floating licenses''''' * PGICDK version ''14.3'' (module <code>pgicdk-14.3</code>): '''''2 permanent network-floating licenses''''' * PGICDK version ''13.10'' (module <code>pgicdk-13.10</code>): '''''2 permanent network-floating licenses''''' * PGICDK version ''12.4'' (module <code>pgicdk-12.4</code>): '''''2 permanent network-floating licenses''''' * PGICDK version ''8.0'' (module <code>pgicdk-8.0</code>): '''''10 permanent network-floating licenses''''' =Use= Initialize an enviroment and run particular program (pgcc, pgCC, pgf77, pgf90, pghpf, pgdbg, pgprof) in accordance with its documentation. <source lang="bash"> module add pgicdk-14.3 # PGI CDK version 14.3 module add pgicdk-13.10 # PGI CDK version 13.10 module add pgicdk-12.4 # PGI CDK version 12.4 module add pgicdk-8.0 # PGI CDK version 8.0 </source> === Programs and environment variables === Quick overview of the most used programs in module to ease you the searching in man pages. {|class="tabulka" |- !Program !Description !Corresponding environment variables !Notes |- | pgcc | preprocessor, compiler, assembler and linker for ANSI and K&R C | CC="pgcc", CPP="pgcc -E" | |- | pgCC | preprocessor, compiler, assembler and linker for C++ | CXX="pgCC", CXXCPP="pgCC -E" | |- | pgc++ | preprocessor, compiler, assembler and linker for C++, compatible with GNU C++ compilers | CXX="pgc++", CXXCPP="pgc++ -E" | |- | pgf77 | preprocessor, compiler, assembler and linker for Fortran 77 | F77="pgf70" | |- | pgf90, pgf95, pgfortran | preprocessor, compiler, assembler and linker for Fortran 90/95 | F90="pgf90", FC="pgf90" | names pgf90 and pgf95 are aliases for pgfortran |- |} =Documentation= Documentation is available on [http://www.pgroup.com/resources/docs.htm producer's WWW server] or locally in the relevant directories: * version 14.9: <code>/software/pgicdk-14.9/linux86-64/14.9/doc/</code> * version 14.3: <code>/software/pgicdk-14.3/linux86-64/14.3/doc/</code> * version 13.10: <code>/software/pgicdk-13.10/linux86-64/13.10/doc/</code> * version 12.4: <code>/software/pgicdk-12.4/linux86-64/12.4/doc/</code> * version 8.0: <code>/software/pgicdk-8.0/linux86-64/8.0/doc</code> =Licence= Every user can use this program on PC clusters of MetaCentrum. =Supported platforms= Linux =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= URL: http://www.pgroup.com/products/cdkindex.htm [[Category:Applications]] [[Category:English]] 8ltosextvjdp7pea577u2xcevd891pd 0 188 5073 4850 2016-05-13T09:13:28Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Python|Česká verze--> ===Description=== Python is a general-purpose high-level programming language. You may utililize its additional libraries like Scipy, Numpy a Scientific Python, which are also installed in MetaCentrum. ===Use=== Initialize environment (module add "module_name"), substitute "module_name" by: *'''python-3.4.1 (modul <code>python-3.4.1-gcc a python-3.4.1-intel</code>)''' *python-2.7.10 (modul <code>python-2.7.10-gcc</code> <code>python-2.7.10-intel</code>) *python-2.7.6 (modul <code>python-2.7.6-gcc a python-2.7.6-intel</code>) *python-2.7.5 (module <code>python-2.7.5</code>) *python-2.6.2 (module <code>python-2.6.2</code>) *python-2.5 (module <code>python-2.5</code>) Some python libraries are also available as modules: <source lang="bash"> module add numpy-1.7.1-py2.7 module add scipy-0.12.0-py2.7 module add pyfits-3.1.2-py2.7 module add sklearn-0.14.1-py2.7 </source> ===Documentation=== Documentation is available on [http://www.python.org/doc/ WWW server] of program. ===Licence=== Every user may use it. ===Supported platforms=== Linux ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://www.python.org/ [[Kategorie:Applications]] tvuv9o4lsokq7s79jcylev7coh9iv3e 0 189 5151 4869 2016-05-13T09:14:37Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--SICStus_Prolog|Česká verze--> ===Description=== SICStus is built around a high performance implementation of logic programming language Prolog engine. SICStus Prolog provides two modes, first according ISO/IEC 13211-1 standard, the other is compatible with C-Prolog and Quintus Prolog. SICStus Prolog provides constraint programming with several constraint solvers ===Use=== Initialize enviroment (module add sicstus-4.0.8 nebo module add sicstus-4.1.2) and run program "sicstus". ===Documentation=== Documentation is available on [http://www.sics.se/isl/sicstuswww/site/documentation.html WWW server] of producer or locally in directory /software/sicstus-4.0.8/doc/. ===Licence=== Please contact administrator, if you are interested in this program. ===Supported platforms=== Linux ===Program administrator=== [mailto:hanka@fi.muni.cz Hana Rudová] ===Homepage=== URL: http://www.sics.se/isl/sicstuswww/site/index.html [[Kategorie:Applications]] amxg6jsbxyoxkl10zwq4hqekoh2hryl 0 193 1915 497 2016-04-30T07:38:22Z Root 1 wikitext text/x-wiki For adding a new application use [[format for application description]]. ===Computational chemistry / Molecular modelling=== Computational chemistry methods serve for computational simulations of real biological behaviour and chemical reactions and processes. * [[ABINIT application|ABINIT]] * [[Amber application|Amber]] * [[Autodock Vina application|Autodock Vina]] * [[Babel application|Babel]] * [[BEAST application|BEAST]] * [[CP2K application|CP2K]] * [[CTHYB application|CTHYB]] * [[DAMASK application|DAMASK]] * [[deMon application|deMon]] * [[Gamess application|Gamess]] * [[Gaussian-GaussView application|Gaussian-GaussView]] * [[Gromacs application|Gromacs]] * [[LAMMPS application|LAMMPS]] * [[Molden application|Molden]] * [[MolPro application|MolPro]] * [[MOPAC application|MOPAC]] * [[NAMD application|NAMD]] * [[NWChem application|NWChem]] * [[Orca application|Orca]] * [[Open3DQSAR application|Open3DQSAR]] * [[PC GAMESS-Firefly application|PC GAMESS-Firefly]] * [[Picard-1.80 application|Picard Tools]] * [[Phonopy application|Phonopy]] * [[Smartflux (EddyPro) application|Smartflux (EddyPro)]] * [[Tinker application|Tinker]] * [[Turbomole application|Turbomole]] * [[VASP application|VASP]] * [[VinaLC application|VinaLC]] ===Structural biology, biology and bioinformatics=== Structural biology is a branch of molecular biology, biochemistry, and biophysics concerned with the molecular structure of biological macromolecules, especially proteins and nucleic acids, how they acquire the structures they have, and how alterations in their structures affect their function. * [[ABySS application|ABySS]] * [[ACT application|ACT]] * [[Affymetrix Power Tools application|Affymetrix Power Tools]] * [[Allpaths application|Allpaths]] * [[Amos application|Amos]] * [[Artemis application|Artemis]] * [[ATSAS application|ATSAS]] * [[Bam2fastq application|Bam2fastq]] * [[Bamtools application|Bamtools]] * [[BayArea_applicaton|BayArea]] * [[Bedtools application|Bedtools]] * [[Bfast application|Bfast]] * [[Bioperl application|Bioperl]] * [[Blast application|Blast]] * [[Bowtie application|Bowtie]] * [[Bucky application|BUCKy]] * [[Cap3-PCap application|Cap3-PCap]] * [[Canu application|Canu]] * [[CD-HIT application|CD-HIT]] * [[CLUMPP application|CLUMPP]] * [[ClustalOmega application|ClustalOmega]] * [[ClustalWX application | ClustalWX]] * [[Clview application| Clview]] * [[CS-Rosetta application|CS-Rosetta]] * [[Cufflinks application|Cufflinks]] * [[DataAnalysis application|DataAnalysis]] * [[dDocent application|dDocent]] * [[DIYABC application|DIYABC]] * [[Edena application|Edena]] * [[Emboss application|Emboss]] * [[ERNE application|ERNE]] * [[eXpress application|eXpress]] * [[Fasta application|Fasta]] * [[FastTree application|FastTree]] * [[FastQC application|FastQC]] * [[Fastx application|Fastx]] * [[FLASH application|FLASH]] * [[FMRIB Software Library (FSL) application|FMRIB Software Library (FSL)]] * [[Freesurfer application|Freesurfer]] * [[FPC application|FPC]] * [[GARLI application|GARLI]] * [[Galaxy application|Galaxy]] * [[GenomeTools application|GenomeTools]] * [[GenomeAnalysisTK (GATK) application|GenomeAnalysisTK (GATK)]] * [[GERP application|GERP]] * [[GMAP (GSNAP) application|GMAP (GSNAP)]] * [[GPAW application|GPAW]] * [[Guidance application|Guidance]] * [[hashrf application|hashrf]] * [[HMMER application|HMMER]] * [[IDBA application|IDBA]] * [[Infernal_applicaton|Infernal]] * [[Irys_applicaton|Irys]] * [[Jellyfish application|Jellyfish]] * [[jvarkit application|jvarkit]] * [[Lagrange application|Lagrange]] * [[Last application|Last]] * [[LASTZ application|LASTZ]] * [[LDDist application|LDDist]] * [[GenomeTools application|LTRdigest]] * [[Maker application|Maker]] * [[Masurca application|Masurca]] * [[Minia application|Minia]] * [[Migrate application|Migrate]] * [[Mira application|Mira]] * [[MIReNA application|MIReNA]] * [[Meraculous application|Meraculous]] * [[MP-EST_application|MP-EST]] * [[MrBayes application|MrBayes]] * [[Mugsy application|Mugsy]] * [[MUMmer application|MUMmer]] * [[Muscle application|Muscle]] * [[Newick Utilities application|Newick Utilities]] * [[Novoalign|Novoalign]] * [[Oases application|Oases]] * [[OMA application|OMA]] * [[PASHA application|PASHA]] * [[PhyBin application|PhyBin]] * [[Phycas application|Phycas]] * [[PhyloBayes application|PhyloBayes]] * [[PhyML application|PhyML]] * [[Picard-1.80 application|Picard Tools]] * [[Proovread application| Proovread]] * [[PyMOL application|PyMOL]] * [[QIIME application|QIIME]] * [[Quake application|Quake]] * [[QUEEN application|QUEEN]] * [[Rainbow application|Rainbow]] * [[Ray application|Ray]] * [[RAxML application|RAxML]] * [[raxmlGUI application|raxmlGUI]] * [[RepeatMasker application|RepeatMasker]] * [[RepeatExplorer application|RepeatExplorer]] * [[RevBayes application|RevBayes]] * [[RNA-mrNA application|rNA-mrNA]] * [[SAMtools application|SAMtools]] * [[SaSSY application|SaSSY]] * [[SEECER application|SEECER]] * [[Seqtk application|Seqtk]] * [[SeqTools application|SeqTools]] * [[SGA application|SGA]] * [[SNAP application|SNAP]] * [[SNAPP application|SNAPP]] * [[SOAP_application|SOAP]] * [[SOAPdenovo application|SOAPdenovo]] * [[SOAPdenovo-trans application|SOAPdenovo-trans]] * [[SOAPaligner application|SOAPalign]] * [[SOLpro application|SOLpro]] * [[SPAdes application|SPAdes]] * [[SPIDER application|SPIDER]] * [[SRATools application|SRATools]] * [[Stacks application|Stacks]] * [[Structure application|Structure]] * [[Tassel application|Tassel]] * [[TGICL application|TGICL]] * [[TNT application|TNT]] * [[TopHat application|TopHat]] * [[Trimmomatic application|Trimmomatic]] * [[Trinity application|Trinity]] * [[Trinotate application|Trinotate]] * [[Unafold application|Unafold]] * [[Velvet application|Velvet]] * [[ViennaRNA application|ViennaRNA]] * [[WGS application|WGS]] * [[X-PLOR application|X-PLOR]] * [[YASRA application|YASRA]] ===Technical and material simulations=== MetaCentrum have available huge range of top-level programs from computational fluid dynamics, computational solid mechanics and associated disciplines. * [[ANSYS application|ANSYS]] with LS-DYNA module * [[Espresso application|Espresso]] * [[Fluent application|Fluent]] * [[MSC.Marc application|MSC.Marc]] * [[Open Foam application|Open Foam]] * [[PALM application|PALM]] * [[PIConGPU application|PIConGPU]] * [[WIEN2k application|WIEN2k]] ===Mathematical and statistical modelling=== The common standard mathematical and statistical packages are in MetaCentrum available as well, of course. * [[CPLEX Optimizer application|CPLEX Optimizer]] * [[Maple application|Maple]] * [[Mathematica application|Mathematica]] * [[gridMathematica application|gridMathematica]] * [[GSL application|GSL]] &ndash; GNU Scientific Library * [[Matlab application|Matlab]] * [[Octave application|Octave]] * [[R application|R]] ===Image, video and sound processing=== * [[Blender application|Blender]] * [[ffmpeg application|ffmpeg]] * [[OpenCV application|OpenCV]] * [[SoX application|SoX]] * [[Sparx application|Sparx]] * [[Xmipp application|Xmipp]] ===Development tools and environments=== Moreover, MetaCentrum offers also an access to professional development tools. * [[Apache Ant application|Apache Ant]] * [[GCC application|GCC]] * [[Grace application|Grace]] * [[HDF5 application|HDF5]] * [[INTEL CDK application|Intel CDK]] * [[JDK application|JDK]] * [[Lisp-CMUCL application|Lisp-CMUCL]] * [[Mono application|Mono]] * [[Numpy application|Numpy]] * [[GNU Parallel application|GNU Parallel]] * [[OMNeT++ application|OMNeT++]] * [[PGI CDK application|PGI CDK]] * [[PyPy application|PyPy]] * [[Python application|Python]] * [[Python - modules]] * [[Scientific Python application|Scientific Python]] * [[Scipy application|Scipy]] * [[SICStus Prolog application|SICStus Prolog]] * [[SGI development environment]] * [[wxPython application|wxPython]] * [[Wine application|Wine]] ===Various other tools=== * [[Argus application|Argus]] * [[Cdbfasta application|Cdbfasta]] [[Category:Applications]] 7a2w29d2m9scjn8zgkuqz834lc4uijn 0 194 5220 5037 2016-05-13T09:17:55Z Root 1 wikitext text/x-wiki <!--([[GridMathematica|Česká verze]])--> = Description = GridMathematica is an integrated extension system for increasing the power of your Mathematica licenses. Each gridMathematica Server gives Mathematica users a shared pool of 16 additional network-enabled Mathematica computation kernels for running distributed parallel computations over multiple CPUs. There is no need to change your existing parallel code—just make gridMathematica Server available, and parallel programs can automatically use the additional CPU power. = Availability = '''Available versions:''' * '''gridMathematica version 9: module <code>gridmathematica-9</code>''' * gridMathematica version 8: module <code>gridmathematica-8</code> * the '''gridMathematica version 9''' could be used (together with our Mathematica licenses) by '''all the MetaVO users'''. The usage with external licenses of Mathematica is restricted by the same limitations as for version 8. * the '''gridMathematica version 8''' License Aggreement currently '''restricts the usage''' of gridMathematica in MetaCentrum '''just on the members''' (students, staff) of the '''following universities''': ** '''ČVUT+VŠCHT''' -- Czech Technical University in Prague and Institute of Chemical Technology in Prague ** '''UK''' -- Charles University in Prague ** '''UTB''' -- Tomas Bata University in Zlin ** '''JCU''' -- University of South Bohemia in Ceske Budejovice ** '''VŠB''' -- VŠB-Technical University of Ostrava To allow an access for another academic institutions/universities, it is '''necessary''' for them '''to purchase/rent''' the Wolfram Mathematica product with an appropriate (''Mathematica Site Unlimited'') license. If you '''intend to be allowed to use MetaCentrum's gridMathematica''', enabling you to perform more complex computations within Mathematica, please, '''let us know before the actual purchase/rent'''. = License = We've rented '''15 licenses''' of gridMathematica -- each of the licenses enables you to extend the computational power of Mathematica by 16 computational kernels (i.e., you are allowed to use up to 240 kernels in total). {{Warning | The purchased licenses permit '''just an academic use''' of the program!}} = Usage = Since MetaCentrum does not feature any licence for Wolfram Mathematica controlling kernels, it is possible to use the provided Wolfram gridMathematica just in the following two modes: * '''Interactive use''', using which you run your copy of the Wolfram Mathematica (with a licence of your home institution) and instruct it to connect to remote computational kernels (Wolfram gridMathematica) running within the MetaCentrum infrastructure (these will be licensed by the MetaCentrum license server). * '''Batch use''', using which you have to make the license server of your home institution accessible from the MetaCentrum computational nodes. Once being started, the Wolfram Mathematica then asks your license server to obtain a license for its controlling kernel; the licenses for the computational kernels will be obtained from the MetaCentrum license server. '''''Note:''' The users, whose home institutions do have the Wolfram Mathematica already installed within the MetaCentrum infrastructure (e.g., University of South Bohemia -- JČU), can---once having the relevant module loaded (e.g., <code>mathematica-9-jcu</code> or <code>mathematica-8-jcu</code>)---use both the presented methods automatically without any further configuration.'' <!-- Since the ''number of licenses for simultaneous users' computations is limited to 15'', we suggest you to run the Mathematica frontends (the controlling kernel) using the '''license of your home institution''' (in such a case, just the computational kernels will be licensed from the MetaCentrum pool of licenses). This approach will bring you a '''higher possibility of a free license''' for your Mathematica computations, and thus a '''faster job startup'''. --> {{ Rámeček | text=''The usage of '''parallel/distributed processing in interactive mode''' is briefly illustrated in '''[http://metavo.metacentrum.cz/cs/seminars/2013-11-13-Mathematica_prezentace.avi the presentation video].'''''}} == Interactive use == Since the Mathematica 8/9 currently does not support the scheduling system we employ (Torque), which could prepare a set of required computational kernels (''MathKernels'') for your remote computations in a comfortable and transparent fashion, we have prepared a '''script "<code>math-grid</code>"''', which makes the remote computational kernels' configuration and setup (otherwise ponderous) as '''comfortable and easy to use''' as possible. This script '''starts the computational kernels''' at first, and subsequently '''provides you with a set of commands''', which (after being run within your Mathematica frontend) instruct your controlling kernel to easily connect to and use the previously started remote computational kernels. This approach further allows you to connect to the computational kernels from "hostile" environments (behind firewalls, NAT, etc.) as well as from Mathematica being run on Windows OS without a necessity to have password-less RSH/SSH configured. ===='''Startup steps (depicted in the [http://metavo.metacentrum.cz/cs/seminars/2013-11-13-Mathematica_prezentace.avi presentation video] as well):'''==== * Using a MetaCentrum frontend, '''ask for an interactive job''' specifying the number of nodes/processors you want to use, and the '''availability of the gridMathematica license''': <source lang="bash"> qsub -I -l nodes=X:ppn=Y:... -l gridmath8=Z # the request asks for X nodes, each of them with Y processors # Z ... the number of Mathematica Grid servers required for your job (generally, Z=X*Y) </source> <!-- '''''alternatively''', if you plan to use MetaCentrum license server for the Mathematica controlling kernel:'' <source lang="bash"> qsub -I -l nodes=X:ppn=Y:... -l gridmath8=Z -l math8=1 # X, Y, Z ... see above </source> --> * Subsequently, prepare all the information necessary to connect to the remote computational kernels: ** using the interactive job console, run the following set of commands: <source lang="bash"> module add gridmathematica-9 ## OR use # module add mathematica-9-XXX ## JUST in cases the module for your institution (XXX) exists # start the computational kernels math-grid start # for more details, see: math-grid --help </source> ** copy the output of the ''math-grid'' command into a clipboard <!-- * '''The further steps''' depend on the case -- whether you want to start the Mathematica frontend '''locally''' or '''remotely''': {| class="wikitable" border="1" ! Local run (preferred) ! Remote run |- | --> <!--* Ensure, that you have a '''direct access (without password specification) to the MetaCentrum computational nodes''' from your node (e.g., using the Kerberos system or assymetric cryptography authentication via SSH) ** see more details about the available authentication mechanisms at [http://metavo.metacentrum.cz/cs/docs/login/index.html MetaCentrum pages]. ** alternatively, control your Mathematica terminal frequently, and whenever your password to access the remote nodes is needed, enter it promptly --> * '''Start your local Mathematica''' (<code>Mathematica</code> command for GUI frontend, or <code>math</code> command for a command-line frontend) and open a new worksheet (''notebook'') ** if your home institution does allow to start the Mathematica on the MetaCentrum nodes (i.e., there is a module dedicated for your institution -- e.g., the "mathematica-9-jcu" module for the University of South Bohemia -- see [[Mathematica application]]), or you have performed an explicit configuration of the license server of your home institution (see below), you can start the controlling Mathematica process (i.e., Mathematica frontend) directly from the established interactive job (and subsequently instruct the controlling process to connect to the started gridMathematica computational kernels). <!--| '''Command-line interface:''' * run the Mathematica command-line interface from the interactive job's terminal (run the <code>math</code> command) '''GUI frontend:''' * in the interactive job's terminal, '''export the display''' to your remote node (or '''tunnel it via SSH''' (option "-X")) and run the Mathematica GUI frontend: <pre> $ export DISPLAY=my_node.somewhere.cz:0 $ Mathematica </pre> alternatively <pre> my_node$ ssh -X mathematica_node.cz mathematica_node$ module add gridmathematica-9; Mathematica </pre> |} --> * '''Copy the output''' of the '''''math-grid''''' command (mentioned above) into the notebook and press ''SHIFT+ENTER'' ** Once the commands become executed, there are all the required remote computational kernels prepared for your computations. <!--*** ''Note: Alternatively, the remote kernels' configuration (and startup) can be performed via a Mathematica's GUI: Edit -> Preferences -> tab Parallel -> subtab Remote Kernels''--> ** '''The list of running computational processes/kernels''' can be obtained by entering the ''Kernels[]'' command into the Mathematica worksheet * '''To check the functionality of parallel processing''', enter the following command into the Mathematica's worksheet (and confirm by pressing ''SHIFT+ENTER''): <source lang="bash"> TableForm[ ParallelEvaluate[{$KernelID, $MachineName, $SystemID, $ProcessID, $Version}], TableHeadings -> {None,{"ID","host","OS","process","Mathematica Version"}}] </source> * Once finished, '''do not forget to stop the running kernels''' (command ''CloseKernels[]'') and subsequently to '''stop the interactive job'''. == Batch use == As mentioned above, to allow the batch computations using the provided Wolfram gridMathematica, there is a ''necessity to obtain a licence for the Wolfram Mathematica controlling process (i.e., the frontend)''. If your home institution does allow to start the Mathematica on the MetaCentrum nodes (i.e., there is a module dedicated for your institution -- e.g., the "<code>mathematica-9-jcu</code>" module for the University of South Bohemia), this license is '''available automatically''' for you. '''''OTHERWISE''''', you have to perform an '''explicit configuration''' to make the Mathematica controlling process asking for a license the license server that is '''legally available for you''' (see the following section). ===='''How to configure Mathematica controlling kernels to use a specific license server:'''==== * in the Mathematica configuration directory (located in your home directory), '''create a file''' <code>$HOME/.Mathematica/Licensing/mathpass</code>, which will contain an '''identification of your home institution's license server''': <source lang="bash"> echo '!mylicenseserver.somewhere.cz' >$HOME/.Mathematica/Licensing/mathpass </source> ** '''''Note:''' The "!" character is essential for a correct server identification.'' * we suggest you to '''make this file unreadable''' for anybody else except you: <source lang="bash"> chmod go= $HOME/.Mathematica/Licensing/mathpass </source> <!--* if the license server of your home institution '''cannot provide you with a free license''' for any reason (temporal unavailability, no free licenses available, etc.), the '''MetaCentrum license server will be used automatically''' (and the job will be started, if there are any free licenses available there) '''''Note:''' If the license server of your home institution is not accessible from outside environment at all (e.g., a firewall being used), please, ignore the above instructions -- the MetaCentrum license server will be used for your computations.'' --> ===='''Startup steps:'''==== * Copy the Mathematica's worksheet containing the intended computation (file '''*.nb''') to a MetaCentrum frontend (and save it to a shared filesystem, for example) ** '''Insert the following lines''', performing an initialization and releasing of parallel environment at computational nodes, at the beginning/end of your file: <source lang="bash"> << "! math-grid start" ... Your parallel computation ... CloseKernels[] </source> ** see the example file [https://meta.cesnet.cz/pubwiki/images/8/84/Example.nb example.nb] * Subseqently, prepare a '''startup script''' for your job (named, e.g., ''startup_script.sh''): <source lang="bash"> #!/bin/bash # module add mathematica-9 ## use JUST in cases you have explicitely configured a license server in your $HOME # module add mathematica-9-XXX ## use JUST in cases the module for your institution (XXX) exists module add gridmathematica-9 math -run <path_to_your_Mathematica_file # ALTERNATIVELY (without implicit outputs): math -script path_to_your_Mathematica_file </source> * '''Submit the startup script''' together with a specification of requested resources (number of computational nodes/processors) and gridMathematica license availability: <source lang="bash"> qsub -l nodes=X:ppn=Y:... -l gridmath8=Z startup_script.sh # request for X nodes, each of them with Y processors # Z ... the number of Mathematica Grid servers required for your job (generally, Z=X*Y) </source> <!-- '''''alternatively''', if you plan to use MetaCentrum license server for the Mathematica controlling kernel:'' <source lang="bash"> qsub -I -l nodes=X:ppn=Y:... -l gridmath8=Z -l math8=1 # X, Y, Z ... see above </source> --> = Documentation = * [http://www.wolfram.com/gridmathematica/ Wolfram official pages] * [http://www.wolfram.com/support/learn/ Wolfram Learning Center] * [http://www.wolfram.com/legal/agreements/wolfram-mathematica.html License Aggreement] = Program administrator = Tomáš Rebok (rebok at ics.muni.cz) = Useful hints = ====Using gridMathematica for parallel runs of computations==== * for example, if you have a serial computation, which you want to be run multiple times in parallel (based on the available cores), create a common '''''master-slave''''' computation as follows: ** the ''slave'' script will perform the computation ** the ''master'' script will run the slave script on all the available cores, using the ''ParallelEvaluate'' function ** '''example of such a computation''' is available for download [[Média:Mathematica-master-slave.zip|HERE]] = Useful links = * Mathematica and Maple comparison: http://www.wolfram.com/mathematica/compare-mathematica/compare-mathematica-and-maple.html ''Note.: If you have some tips for useful Mathematica-related pages, please, let us know...'' [[Kategorie:Applications]] tbupaf1as3fl3tcpnsrjvm1kzlo9lb0 0 195 1922 1921 2016-04-30T07:49:15Z Root 1 wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[:Category:Applications|Available applications]]|}} }} The used ''modullar subsystem'' provides a user interface to modifications of user environment, which are necessary for running the requested applications. It allows to configure the user environments independently on the server's platform or files' locations; thus, the user is not required to perform (usually complicated) manual environment modifications in order to allow the use of requested applications. {{GreenBox|The [[Frontend|MetaCentrum frontends]] may provide a slightly different modullar applications' equipment, since these are intended to be used for interactive job preparations, input/output data examinations, etc. If a precisely identical environment is necessary for an interactive work (i.e., one wants to test a job run, which performs some compilations), please, ask the scheduler for an [[How to compute/Interactive jobs|interactive job]] and subsequently perform all the intended operations within it.}} = Availability = The ''modules'' module = Usage = * A '''list of available modules''' (applications) can be obtained by the following command (run on any MetaCentrum node): <source lang=bash> $ module available </source> * A '''requested module''' (available in the list) can be '''loaded into a user environment''' using the following command: <source lang=bash> $ module add <modulename> # e.g., module add matlab </source> '''''Note:''' The command integrates the chosen application into the user environment, including search PATH update, MANPATH update, and necessary initialization(s).'' * The '''list of already loaded modules''' can be obtained using the command <source lang=bash> $ module list </source> * To '''remove a previously loaded module''', one can write the following command: <source lang=bash> $ module unload <modulename> # OR module del <modulename> # e.g., module unload matlab </source> * To '''use the modullar subsystem within job startup scripts''', it could be necessary to initialize the modullar subsystem itself at first; subsequently, the requested application can be loaded into the environment: <source lang=bash> source /software/modules/init module add <modulename> </source> = Documentation = * [http://modules.sourceforge.net/ official Environment Modules Project site] * [http://modules.sourceforge.net/man/module.html ''module'' manpages] (<code>man module</code>) * [http://modules.sourceforge.net/man/modulefile.html ''modulefile'' manpages] (<code>man modulefile</code>) = License = Open source (GPLv2). = Supported platforms = UNIX systems. = Program administrator = [mailto:salvet@ics.muni.cz Zdeněk Salvet] [[Category:Applications]] pf5isgye6y3crd3nh8j7zjy5jwva0jc 0 196 5114 4992 2016-05-13T09:14:05Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Mono|Česká verze--> ===Description=== Mono is a software platform designed to allow developers to easily create cross platform applications. Sponsored by Xamarin, Mono is an open source implementation of Microsoft's .NET Framework based on the ECMA standards for C# and the Common Language Runtime. ===Availability=== Version 2.10.2 ===Use=== Run "module add mono-2.10.2" ===How to=== Hello world, hello.cs <source lang="c"> using System; public class HelloWorld { static public void Main () { Console.WriteLine ("Hello Mono World"); } } </source> Compile and run by <source lang="bash"> gmcs hello.cs mono hello.exe </source> ===Documentation=== http://www.mono-project.com/Start ===Licence=== GPL ===Supported platforms=== Linux ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.mono-project.com/ [[Kategorie:Applications]] b6ns7th5u9n5vma8ugje6yd2vdz2usk 0 200 5163 4954 2016-05-13T09:14:47Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Velvet|Česká verze--> = Description = Velvet is frequently used assembler in many sequencing projects that are focused on de novo assembly from NGS technology data. Velvet is capable of handling paired and also single reads. It permits the user to set many of assembly conditions. = Availability = Versions 1.2.03, 1.2.07 and 1.2.10. Freely available to users. Modules: velvet-1.2.03 velvet-1.2.07 velvet-1.2.10 VelvetOptimizer 2.2.5 is available too (module velvetopt-2.2.5). = Licence = [http://www.gnu.org/licenses/gpl-3.0.txt GPLv3] = Use = Initialize environment with command: module add velvet-1.2.10 = Documentation = Documentation is available at Velvet official website. Program help (manual) is also part of the package. = Program administrator = * Jan Ištvánek -- Laboratory of molecular plant genetics, PřF MU (janites at mail.muni.cz, i.jan at centrum.cz) * Tomáš Rebok (rebok at ics.muni.cz) = Homepage = URL: http://www.ebi.ac.uk/~zerbino/velvet/ VelvetOptimizer: http://bioinformatics.net.au/software.velvetoptimiser.shtml [[Category:Applications]] [[Category:English]] ileasfc4qq2gxj11bl3ucb793ja6taa 0 205 5157 5113 2016-05-13T09:14:42Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--DataAnalysis|Česká verze--> ===Description=== The GS Data Analysis Software package includes the tools to investigate complex genomic variation in samples including de novo assembly, reference guided alignment and variant calling, and low abundance variant identification and quantification. * GS De Novo Assembler - Point-and-click genome and transcriptome assembly * GS Reference Mapper - Reference-guided alignment and variation detection for resequencing projects * GS Amplicon Variant Analyzer - High-sensitivity variation detection from amplicon-based samples ===Availability=== Currently in version 2.6 ===Use=== Add modul ("module add dataanalysis-2.6") and than you can run programs (e.g. newbler, gsAssembler, gsAmplicon) see "ls /software/dataanalysis-2.6/bin/" for list of programs. ===Documentation=== http://454.com/products/analysis-software/index.asp ===Licence=== http://454.com/contact-us/agreement.asp ===Supported platforms=== Linux ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.454.com http://454.com/products/analysis-software/index.asp [[Kategorie:Applications]] 19789gwaij8p37yp2nbokk6jc1gjmkr 0 206 5021 4866 2016-05-13T09:12:39Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Espresso|Česká verze--> ===Description=== Quantum espresso is an integrated suite of Open-Source computer codes for electronic-structure calculations and materials modeling at the nanoscale. It is based on density-functional theory, plane waves, and pseudopotentials. ===Availability=== Version 4.3.1, 5.0 ===Use=== Add one of modules by module add MODULNAME * espresso-5.0 * espresso-5.0-thread (openmp version) * espresso-5.0-mpi (openmpi version) * espresso-4.3.1-thread (openmp version) * espresso-4.3.1-mpi (openmpi version) * espresso-4.3.1-mpich2 (mpich2 version) in the case of using mpi version, add explicitly mpi modul after adding of espresso, e.g. for espresso-5.0-mpi module add espresso-5.0-mpi and run (MPI version) mpiexec pw.x For testing examples e.g. in ''/software/espresso-5.0/single/PW/example'' is necessary in file ''run_example'' to change path for ''environmental_variables'' e.g. to ''/software/espresso-5.0/single/environmental_variables''. Module gipaw is working only in 4.3.1 thread and mpich2 versions. If your run assumes vdW_kernel_table, you can generate it by generate_vdW_kernel_table.x than you should copy it into $PSEUDO_DIR (folder that is set after adding of module). ===Documentation=== http://www.quantum-espresso.org/?page_id=40 ===Licence=== GNU General Public License ===Supported platforms=== amd64 ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.quantum-espresso.org/ [[Kategorie:Applications]] k6vxj6b31j9klgk1jl4o93f15tbpk2t 0 207 5076 4940 2016-05-13T09:13:31Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--NWChem|Česká verze--> ===Description=== NWChem is a computational chemistry package that has been developed by the Molecular Sciences Software group of the Environmental Molecular Sciences Laboratory at the Pacific Northwest National Laboratory. It provides many methods to compute the properties of molecular and periodic systems using standard quantum mechanical descriptions of the electronic wavefunction or density. It also has the capability to perform classical molecular dynamics and free energy simulations. ===Availability=== * '''NWChem 6.3.2''' (module <code>NWChem-6.3.2</code>), 64-bit version, compiled with: ** Intel Compilers and Intel MKL libraries support (BLAS, LAPACK, ScaLAPACK) ** MPI support (OpenMPI) ** CUDA support ** all the included modules ** large files support ** Python support/interconnection * '''NWChem 6.0''' (module <code>NWChem-6.0</code>), 64-bit version, compiled with: ** MPI support (OpenMPI) ** all the included modules ** large files support ** Python support/interconnection ===Use=== '''1. Running the application -- ''Interactive mode:''''' * ask the scheduler for an interactive job having a desired number of nodes and processors (<code>ppn</code> attribute) reserved <source lang="bash"> $ qsub -I -l nodes=X:ppn=Y -l mem=Zg </source> :''Note: Do not forget to apppropriately set the amount of requested memory (<code>mem</code> attribute) and/or another job requirements.'' * load the NWChem module <source lang="bash"> $ module add NWChem-6.3.2 $ ulimit -s unlimited </source> * prepare the input file and run the computation ** '''single-process single-threaded run''' ''(will use exactly 1 CPU on a single node)'' <source lang="bash"> $ nwchem <inputfile> </source> :* '''single-process multi-threaded run''' ''(will use exactly Y CPUs on a single node)'' <source lang="bash"> $ export OMP_NUM_THREADS=$PBS_NUM_PPN $ nwchem <inputfile> </source> :* '''multi-process parallel/distributed run''' ''(will use X*Y CPUs on X nodes)'' <source lang="bash"> $ mpirun nwchem <inputfile> </source> {{ Warning | It the <code>nwchem</code> shows an error message <code>Unable to open nwchem.nw --- appending .nw</code> '''in the case of parallel/distributed run''', '''create an empty file''' named <code>nwchem.nw</code> in the computation working directory. This should resolve the problem.}} '''2. Running the application -- ''Batch mode:''''' * prepare the job description script -- use a general skeleton supplemented by the following lines: <source lang="bash"> ... # load the NWChem module module add NWChem-6.3.2 # increase the stack size ulimit -s unlimited # run the computation (choose one of the following) export OMP_NUM_THREADS=$PBS_NUM_PPN # use, when multi-threaded single-node version is desired nwchem mydata.nw >mydata.out # single-process single-node version mpirun nwchem mydata.nw >mydata.out # parallel/distributed version ... </source> * pass the job description file to the scheduler together with (at least) the requested number of processors and requested amount of memory <source lang="bash"> $ qsub -l nodes=X:ppn=Y -l mem=Zg mydecriptionscript.sh </source> {{ Warning | It the <code>nwchem</code> shows an error message <code>Unable to open nwchem.nw --- appending .nw</code> '''in the case of parallel/distributed run''', '''create an empty file''' named <code>nwchem.nw</code> in the computation working directory. This should resolve the problem.}} ===Documentation=== * version '''6.3.2''': http://www.nwchem-sw.org/index.php/Release62:NWChem_Documentation * version '''6.0''': http://www.nwchem-sw.org/index.php/NWChem_Documentation ===Licence=== open-source under the terms of the [http://www.opensource.org/licenses/ecl2.php Educational Community License version 2.0 (ECL 2.0)] ===Supported platforms=== amd64 ===Program manager=== Tom Rebok ([mailto:meta@cesnet.cz meta@cesnet.cz]) ===Homepage=== http://www.nwchem-sw.org [[Kategorie:Applications]] ntzcqlkcp4cn0e9tfnfqw84328gc6d6 0 208 4857 4803 2016-05-13T09:09:24Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--raxmlGUI|Česká verze--> ===Description=== A userfriendly graphical front-end for phylogenetic analyses using RAxML (Stamatakis, 2006). ===Availability=== Version 1.3.1 ===Use=== Add module and run application module add raxmlGUI-1.3.1 raxmlGUI.py Linux version do no allow muptiple threads. ===Documentation=== http://sourceforge.net/projects/raxmlgui/ ===Licence=== GNU General Public License version 2.0 (GPLv2) ===Supported platforms=== amd64 ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://sourceforge.net/projects/raxmlgui/ [[Kategorie:Applications]] 4isd746hlxw29189qleoorlalcpon46 0 210 5199 5083 2016-05-13T09:15:19Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Blast|Česká verze--> =Description= The NCBI Basic Local Alignment Search Tool (BLAST) finds regions of local similarity between sequences. The program compares nucleotide or protein sequences to sequence databases and calculates the statistical significance of matches. BLAST can be used to infer functional and evolutionary relationships between sequences as well as help identify members of gene families. =Availability= * '''Blast+''' version 2.2.27 and 2.2.29 ** module <code>blast+-2.2.27</code> * ''Blast+'' version 2.2.26 ** module <code>blast+-2.2.26</code> * '''Blast''' version 2.2.26 ** module <code>blast-2.2.26</code> ===BlastDB=== We maintain a '''local copy of several Blast databases''' &ndash; see the <code>/storage/projects/BlastDB</code> directory. All available databases are described [ftp://ftp.ncbi.nlm.nih.gov/blast/documents/blastdb.html at NCBI web]. We mirror all of them. Last update of the offline copy is 3.3.2015. If you need to update DBs or to add some new, please contact the module maintainer. =Use= module add blast-2.2.26 # Blast module add blast+-2.2.27 # Blast+ then you may run the Blast binaries (see <code>/software/blast-2.2.26/bin</code>, resp. <code>/software/blastPlus-2.2.26/bin</code> or <code>/software/blast-2.2.27/bin</code>). =Documentation= http://www.ncbi.nlm.nih.gov/books/NBK1762/ =Licence= It was written as part of the author's official duties as a United States Government employee and thus cannot be copyrighted. This software/database is freely available to the public for use. The National Library of Medicine and the U.S. Government have not placed any restriction on its use or reproduction. =Program manager= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://blast.ncbi.nlm.nih.gov [[Category:Applications]] [[Category:English]] m9n0omiupd8ny8irakkp24xjimbt6te 0 211 5197 5183 2016-05-13T09:15:17Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Mira|Česká verze--> ===Description=== MIRA is a whole genome shotgun and EST sequence assembler for Sanger, 454, Solexa (Illumina), IonTorrent data and PacBio (the later at the moment only CCS and error-corrected CLR reads). It can be seen as a Swiss army knife of sequence assembly developed and used in the past 12 years to get assembly jobs done efficiently - and especially accurately. That is, without actually putting too much manual work into finishing the assembly. ===Availability=== Version 3.4.0, 3.9.8dev and 4.0.2 ===Use=== module add mira-4.0.2 mira or other binaries (see Documentation). ===Documentation=== http://sourceforge.net/apps/mediawiki/mira-assembler/index.php?title=Main_Page ===Licence=== GPL licence (currently v2) ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://sourceforge.net/projects/mira-assembler/ [[Category:Applications]] [[Category:English]] kr4pxfix52vss956ebyay27rrlvejya 0 214 5212 5139 2016-05-13T09:15:36Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Oases|Česká verze--> ===Description=== Oases is a de novo transcriptome assembler designed to produce transcripts from short read sequencing technologies, such as Illumina, SOLiD, or 454 in the absence of any genomic assembly. It was developed by Marcel Schulz (MPI for Molecular Genomics) and Daniel Zerbino (previously at the European Bioinformatics Institute (EMBL-EBI), now at UC Santa Cruz). Oases uploads a preliminary assembly produced by Velvet, and clusters the contigs into small groups, called loci. It then exploits the paired-end read and long read information, when available, to construct transcript isoforms. ===Availability=== Version 0.2.08 ===Use=== module add oases-0.2.08 oases # eventually: oases_pipeline.py It also loads [[Velvet application|velvet-1.2.10]]. For details, see [[#Documentation|Documentation]]. ===Documentation=== http://www.ebi.ac.uk/~zerbino/oases/ ===Licence=== GPL 3.0 ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.ebi.ac.uk/~zerbino/oases/ [[Category:Applications]] [[Category:English]] eewbbt5ku69udewishtyhq550lpz6g9 0 215 4995 4906 2016-05-13T09:12:08Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Fasta|Česká verze--> ===Description=== FASTA is a DNA and protein sequence alignment software package first described (as FASTP) by David J. Lipman and William R. Pearson in 1985.[1] Its legacy is the FASTA format which is now ubiquitous in bioinformatics. Although there are a large number of programs in this package, they belong to three groups: (1) Traditional similarity searching programs: fasta36, fastx36, fasty36, tfastx36, tfasty36, ssearch36, ggsearch36, and glsearch36; (2) Programs for searching with short fragments: fasts36, fastf36, tfasts36, tfastf36, and fastm36; (3) A program for finding non-overlapping local alignments: lalign36. Programs that start with fast search protein databases, while tfast programs search translated DNA databases. ===Availability=== Version 36.3.5c ===Use=== module add fasta-36.3.5c fasta36 or other binaries (see Documentation). ===Documentation=== http://faculty.virginia.edu/wrpearson/fasta/ ===Licence=== Free for non-commercial use ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://faculty.virginia.edu/wrpearson/fasta/ [[Kategorie:Applications]] hfigoma987g31aynpecihxsajhqw2vp 0 217 5149 4872 2016-05-13T09:14:36Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--TopHat|Česká verze--> ===Description=== TopHat is a fast splice junction mapper for RNA-Seq reads. It aligns RNA-Seq reads to mammalian-sized genomes using the ultra high-throughput short read aligner Bowtie, and then analyzes the mapping results to identify splice junctions between exons. ===Availability=== Version 2.0.4, 2.0.8 and 2.0.12. Available modules: tophat-2.0.4 tophat-2.0.8 tophat-2.0.12 ===Use=== module add tophat-2.0.4 cuffdiff and other binaries, see Documentation. ===Documentation=== http://tophat.cbcb.umd.edu/ ===Licence=== Boost Software License, Version 1.0 ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://tophat.cbcb.umd.edu/ [[Kategorie:Applications]] [[Kategorie:English]] bgg7hdlw3e25pfximaqfguw2or4mdvw 0 218 4997 4904 2016-05-13T09:12:10Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Cufflinks|Česká verze--> ===Description=== Cufflinks assembles transcripts, estimates their abundances, and tests for differential expression and regulation in RNA-Seq samples. It accepts aligned RNA-Seq reads and assembles the alignments into a parsimonious set of transcripts. Cufflinks then estimates the relative abundances of these transcripts based on how many reads support each one, taking into account biases in library preparation protocols. ===Availability=== Version 2.0.2 and 2.2.1. Available modules: cufflinks-2.2.1 cufflinks-2.0.2 ===Use=== module add cufflinks-2.2.1 cuffdiff and other binaries, see Documentation. For parallel computing is necessary to use the parameter <code>-p <number of processors></code>. You can take the number of reserved processors (specified by qsub command) from the torque system: -p $TORQUE_RESC_PROC ===Documentation=== http://cufflinks.cbcb.umd.edu/ ===Licence=== Boost Software License, Version 1.0 ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://cufflinks.cbcb.umd.edu/ [[Kategorie:Applications]] bj4emaf4tewufdyq8glmt6j74vwcagp 0 219 5159 4853 2016-05-13T09:14:43Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--MUMmer|Česká verze--> ===Description=== MUMmer is a system for rapidly aligning entire genomes, whether in complete or draft form. For example, MUMmer 3.0 can find all 20-basepair or longer exact matches between a pair of 5-megabase genomes in 13.7 seconds, using 78 MB of memory, on a 2.4 GHz Linux desktop computer. MUMmer can also align incomplete genomes; it can easily handle the 100s or 1000s of contigs from a shotgun sequencing project, and will align them to another set of contigs or a genome using the NUCmer program included with the system. ===Availability=== Version 3.23 ===Use=== module add mummer-3.23 mummer or other binaries (see Documentation). ===Documentation=== http://mummer.sourceforge.net/manual/ ===Licence=== Artistic License (see /software/mummer-3.23/LICENSE) ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://mummer.sourceforge.net/ [[Kategorie:Applications]] 7nqizl9v7xl1uutcrvfk540jrp9tfnx 0 226 5001 4984 2016-05-13T09:12:14Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Unafold|Česká verze--> ===Description=== Unified Nucleic Acid Folding and hybridization package. ===Availability=== Version 3.8 ===Use=== module add unafold-3.8 UNAFold.pl or other scripts (see Documentation). ===Documentation=== http://mfold.rna.albany.edu/ ===Licence=== Academic non-commercial see http://mfold.rna.albany.edu/?q=DINAMelt/non-commercial-license ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://mfold.rna.albany.edu/ [[Kategorie:Applications]] c3x2649tfyr0o25y4mv20oym8illv7k 0 230 4795 4746 2016-05-13T09:07:57Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Amos|Česká verze--> ===Description=== The AMOS consortium is committed to the development of open-source whole genome assembly software. The project acronym (AMOS) represents our primary goal -- to produce A Modular, Open-Source whole genome assembler. Open-source so that everyone is welcome to contribute and help build outstanding assembly tools, and modular in nature so that new contributions can be easily inserted into an existing assembly pipeline. This modular design will foster the development of new assembly algorithms and allow the AMOS project to continually grow and improve in hopes of eventually becoming a widely accepted and deployed assembly infrastructure. In this sense, AMOS is both a design philosophy and a software system. ===Availability=== Version 3.1.0 ===Use=== module add amos-3.1.0 bank-transact see Documentation ===Documentation=== http://sourceforge.net/apps/mediawiki/amos/index.php?title=AMOS_Getting_Started ===Licence=== Artistic license ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://sourceforge.net/apps/mediawiki/amos/index.php?title=AMOS [[Kategorie:Applications]] a5m45lyz29f4abxrsnpuw15kwmmuouz 0 234 4811 579 2016-05-13T09:08:22Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki [[ClustalWX|Česká verze--> ===Description=== Clustal X is a windows interface for the ClustalW multiple sequence alignment program. ===Availability=== Version 2.1 ===Use=== module add clustal-wx-2.1 clustalw2 clustalx see Documentation ===Documentation=== http://www.clustal.org/clustal2/ ===Licence=== GPL version 3 ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.clustal.org/clustal2/ [[Kategorie:Applications]] c31yl4ihnr421xsnxkb3molr2iil9rg 0 237 5214 5124 2016-05-13T09:15:39Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Moses|Česká verze--> ===Description=== '''Moses''' is a statistical machine translation system that allows you to automatically train translation models for any language pair. All you need is a collection of translated texts (parallel corpus). An efficient search algorithm finds quickly the highest probability translation among the exponential number of choices. ===Availability=== Available modules: *<code>moses-2.1.1</code>: ''Moses version 2.1.0'', 64-bit, compiled with KenLM language model toolkit and distributed computing (MPI) support, includes GIZA++ and IRSTLM model tools *<code>moses-1.0</code>: ''Moses version 1.0'', 64-bit, compiled with KenLM language model toolkit and distributed computing (MPI) support, includes GIZA++ and IRSTLM model tools *<code>moses-1.0beta</code>: ''Moses version 1.0beta'' (release 20120926), 64-bit, compiled with KenLM language model toolkit ===Use=== '''1. Running the application -- ''Interactive mode:''''' * ask the scheduler for an interactive job having desired number of nodes/processors (<code>nodes</code> and <code>ppn</code> attributes) reserved ** '''''Note.:''' Moses '''version 1.0beta''' (module <code>moses-1.0beta</code>) is compiled '''without the MPI support''' => ask just for a single node (the others won't be used by Moses)'' <source lang="bash"> $ qsub -I -l nodes=X:ppn=Y -l mem=Zg </source> :''Note: Do not forget to apppropriately set the amount of requested memory (<code>mem</code> attribute) and/or another job requirements.'' * load the Moses module <source lang="bash"> $ module add moses-1.0 # moses-1.0beta </source> * prepare the configuration and the input files and run the computation <source lang="bash"> $ moses -f config_file.ini <input_file >output_file # 1-node computation $ mpirun moses -f config_file.ini <input_file >output_file # MPI version $ moses_chart -f config_file.ini <input_file >output_file # 1-node computation $ mpirun moses_chart -f config_file.ini <input_file >output_file # MPI version </source> '''2. Running the application -- ''Batch mode:''''' * prepare the job description script -- use a general skeleton supplemented by the following lines: <source lang="bash"> ... # load the Moses module module add moses-1.0 # moses-1.0beta # run the computation moses -f config_file.ini <input_file >output_file # 1-node computation # mpirun moses -f config_file.ini <input_file >output_file # MPI version # moses_chart -f config_file.ini <input_file >output_file # 1-node computation # mpirun moses_chart -f config_file.ini <input_file >output_file # MPI version ... </source> * pass the job description file to the scheduler together with (at least) the requested number of nodes/processors and requested amount of memory <source lang="bash"> $ qsub -l nodes=X:ppn=Y -l mem=Zg mydescriptionfile.sh </source> :* '''''Note.:''' Moses '''version 1.0beta''' (module <code>moses-1.0beta</code>) is compiled '''without the MPI support''' => ask just for a single node (the others won't be used by Moses)'' ===Documentation=== http://www.statmt.org/moses/manual/manual.pdf ===License=== open-source under the terms of the [http://www.gnu.org/licenses/lgpl-3.0.en.html GNU Lesser General Public License] ===Supported platforms=== amd64 ===Program administrator=== Tom Rebok [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.statmt.org/moses [[Kategorie:Applications]] ohzi565lf549l10vq4vr7a7ye6lc4ko 0 239 5117 4968 2016-05-13T09:14:08Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--IDBA|Česká verze--> ===Description=== IDBA is a practical iterative De Bruijn Graph De Novo Assembler for sequence assembly in bioinfomatics. ===Availability=== Version 0.20 ===Use=== module add ibda-0.20 idba see Documentation. ===Documentation=== http://i.cs.hku.hk/~alse/hkubrg/projects/idba/ ===Licence=== Not specified. ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://i.cs.hku.hk/~alse/hkubrg/projects/idba/ [[Kategorie:Applications]] oy0t0biyl3ff676i3a0bzshpv9x63o8 0 240 5105 4787 2016-05-13T09:13:58Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--SPAdes|Česká verze--> ===Description=== SPAdes stands for St. Petersburg genome assembler. It is intended for both standard (multicell) and single-cell MDA bacteria assemblies. ===Availability=== Version 2.2.1, 2.5.1, 3.1.0 and 3.6.0(intel compiler + original binaries) ===Use=== module add spades-3.6.0 spades.py see Documentation ===Documentation=== http://spades.bioinf.spbau.ru/release2.5.1/manual.html ===Licence=== GPL version 2. ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://bioinf.spbau.ru/en/spades [[Kategorie:Applications]] a03x9biajs1s9yryyenk0qtvc8grkal 0 241 4841 4821 2016-05-13T09:09:00Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--TotalView|Česká verze--> = Description = '''TotalView''' is the most commonly used debugging tool providing an interactive graphical user interface for debugging both single-threaded and parallel/distributed applications. It supports most common programming models and libraries such as MPI, OpenMP, Threads, PVM, HPF, Shmem and fork/exec, including the GPU/CUDA debugging. Besides the basic functionality available in most debuggers, the TotalView provides advanced memory-debugging functionality (through the ''MemoryScape'' module), non-interactive debugging of batch jobs (via the ''TVScript'' module), and a unique opportunity to perform reverse-debugging possibilities (debugging "from mistakes to causes" -- via the ''ReplayEngine'' module). = Availability = ''TotalView'' is available in the versions: * '''version 8.12''' via module <code>totalview-8.12</code> * ''version 8.10'' via module <code>totalview-8.10</code> ''Licensing:'' * the application is licensed for '''64 simultaneously debugged processes ''(so-called tokens)''''' {{Warning | The purchased licenses permit '''just an academic use''' of the program!}} = Use = Tunnelling the graphical environment, log into a node, where you want to run the ''TotalView'': <source lang="bash"> $ ssh -X skirit.metacentrum.cz skirit$ module add totalview-8.12 skirit$ totalview </source> {{Application with GUI}} = Documentation = The TotalView documentation is available either online at [http://www.roguewave.com/support/product-documentation/totalview.aspx producer's webpage], or locally in the program directory (<code>/software/totalview-8.12/doc/</code>, or <code>/software/totalview-8.10/doc/</code>). = Program administrator = * Tomáš Rebok (rebok at ics.muni.cz) = Homepage = URL: http://www.roguewave.com/products/totalview.aspx [[Kategorie:Applications]] [[Category:English]] dlol14sv6nmgi3ssim7la1jfrs944b0 0 242 5222 5221 2016-05-13T09:18:54Z Root 1 /* Description */ wikitext text/x-wiki <!--([[Allinea_DDT|Česká verze]])--> = Description = Allinea DDT is a debugging tool from Allinea Inc., designed for debugging parallel/distributed programs using OpenMP/MPI libraries. Its primary aim is to provide a powerful and user-friendly graphical user interface, together with unique features such as so-called ''Parallel Stack Viewer'' (providing a global overview of the status of all the debugged application's threads/processes). Unlike its competitor ([[TotalView application]]), the Allinea DDT provides ''high scalability'' -- it is able to debug applications consisting of more than 200,000 concurrently running processes/threads. = Availability = ''Allinea DDT'' is available in the versions: * '''4.2''' via the module <code>allineaDDT-4.2</code> or via module <code>allineaDDT</code> * ''4.1'' via the module <code>allineaDDT-4.1</code> * ''3.2'' via the module <code>allineaDDT-3.2</code> =Licensing= * the debugger is licensed for '''32 simultaneously debugged processes''' * till 10/2013, the DDT may be used on all the MetaCentrum clusters; after this time period, the DDT may be used just on the clusters located at Botanická 68a, Brno {{Warning | The purchased licenses permit '''just an academic use''' of the program!}} = Use = Tunnelling the graphical environment, log into a node, where you want to run the ''Allinea DDT'': <source lang="bash"> $ ssh -X skirit.metacentrum.cz skirit$ module add allineaDDT skirit$ ddt # ddt-mpirun, ... </source> {{Application with GUI}} = Documentation = The Allinea DDT documentation is available: '''version 4.2:''' * either online at [http://content.allinea.com/downloads/userguide.pdf producer's webpage], or locally in the program directory (<code>/software/allineaDDT/4.2/doc/</code>). ''version 4.1:'' * either online at [http://content.allinea.com/downloads/userguide.pdf producer's webpage], or locally in the program directory (<code>/software/allineaDDT-4.1/doc/</code>). ''version 3.2:'' * either online at [http://www.allinea.com/Portals/90122/docs/user-guides-and-technical-docs/allinea-ddt-3.2-user-guide-july-2012.pdf producer's webpage], or locally in the program directory (<code>/software/allineaDDT-3.2/doc/</code>). = Program administrator = * Tomáš Rebok (rebok at ics.muni.cz) = Homepage = URL: http://www.allinea.com/products/ddt/ [[Kategorie:Applications]] aqkone3ay9ky5vbbiz7s5c1wbt9kkat 0 249 5132 5048 2016-05-13T09:14:21Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--STDTOOLS|Česká verze--> ===Description=== '''STDTOOLS''' is a package for indexing and searching a collection of weighted finite state automata. Although it is designed with the Spoken Term Detection (STD) task in mind, it can be used for realizing a variety of information retrieval tasks with little to no modification. It is built upon [[OpenFst | OpenFST]], a library for constructing, combining, optimizing, and searching weighted finite-state transducers (FSTs). These utilities construct and search timed factor transducers. A factor transducer is an efficient inverted index of a collection of finite state automata. This version of the STDTOOLS package embeds timing of the states of the input automata into the factor transducer arc weights. Timing information is supplied in the form of state potentials (auxiliary cost information about states). For a detailed description of the timed factor transducer structure as well as the algorithm used for its construction: http://busim.ee.boun.edu.tr/~dogan/files/dogancan-TASLP-revision.pdf ===Availability=== '''Module <code>stdtools-0.2</code>:''' ''STDTOOLS version 0.2'', 64-bit, ... ===Use=== stdindex, stdsearch, stdprocess - spoken term detection utilities '''Running the application''' This is only a preview of the work with these tools. For details see appropriate man pages. * PREPROCESS - reads a list of FST-IN POTENTIAL-IN FST-OUT POTENTIAL-OUT quadruplets from file in.list or standard input. It processes (pruning, arc clustering, etc.) the input files and writes them to the specified output files. <source lang="bash"> $ stdprocess in.list out.list </source> Input Format: in1.fst in1.pot out1.fst out1.pot in2.fst in2.pot out2.fst out2.pot * INDEX - reads a list of FST POTENTIAL tuples from file in.list or standard input. It writes a timed factor transducer to file index.fst or standard output. <source lang="bash"> $ stdindex in.list index.fst </source> Input Format of the in.list file: in1.fst in1.pot in2.fst in2.pot * SEARCH - reads index from file index.fst and queries from file query.far or standard input. It writes search results to file out.list or standard output. <source lang="bash"> $ stdsearch index.fst query.far out.list </source> ===Documentation=== * locally using the following commands: ** <code>$ man stdtools</code> ** <code>$ man fstar</code> ===License=== open-source, distributed under the terms of the [http://www.apache.org/licenses/LICENSE-2.0 Apache license]. ===Supported platforms=== amd64 ===Program administrator=== Jan Vavruška (vavruska (at) kky.zcu.cz) ===Homepage=== http://www.busim.ee.boun.edu.tr/~dogan/research.html [[Kategorie:Applications]] hm2lut9x2jlilin8213r9jw88i00af0 0 253 5026 4941 2016-05-13T09:12:42Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Coreutils|Česká verze--> ===Description=== The '''GNU Core Utilities''' are the basic file, shell and text manipulation utilities of the GNU operating system. Previously these utilities were offered as three individual sets of GNU utilities, ''Fileutils'', ''Shellutils'', and ''Textutils''. Those three have been combined into a single set of utilities called ''Coreutils''. These are the core utilities which are expected to exist on every node of the infrastructure---the module is available because of advanced functionality of the latest version (e.g., parallel processing by the <code>sort</code> command). ===Availability=== The Coreutils are available in the version '''8.20''' via module <code>coreutils-8.20</code>. ===Use=== Load the module ("<code>module add coreutils-8.20</code>") and run the desired program (e.g., <code>sort</code>). ===Documentation=== The documentation is available [http://www.gnu.org/software/coreutils/manual/ on-line] or locally via the <code>man</code> utility (e.g., <code>man sort</code>). ===Supported platforms=== Linux ===Program administrator=== [mailto:meta@cesnet.cz Tom Rebok] ===Homepage=== URL: http://www.gnu.org/software/coreutils/ [[Kategorie:Applications]] dbzanyohpuxcmk4fikejj92zdibkl7z 0 254 5161 4792 2016-05-13T09:14:45Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Bam2fastq|Česká verze--> ===Description=== The BAM format is an efficient method for storing and sharing data from modern, highly parallel sequencers. While primarily used for storing alignment information, BAMs can (and frequently do) store unaligned reads as well. There are a growing number of general-purpose SAM/BAM manipulation programs, including samtools, Picard, and Bamtools. This tool is not intended to duplicate the complex suite of tasks those programs perform. Rather, it is simply intended to extract raw sequences (with qualities). We envision this tool being primarily useful to those wishing to duplicate or extend previous analyses. ===Availability=== Version 1.1.0 ===Use=== module add bam2fastq-1.1.0 bam2fastq ===Documentation=== http://www.hudsonalpha.org/gsl/software/bam2fastq.php ===Licence=== Apache License Version 2.0 ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.hudsonalpha.org/gsl/software/bam2fastq.php [[Kategorie:Applications]] ei7xh5jet905kqobc69ejt7vg2w4m3z 0 255 4959 4825 2016-05-13T09:11:26Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--TGICL|Česká verze--> ===Description=== TGI Clustering tools (TGICL): a software system for fast clustering of large EST datasets This package automates clustering and assembly of a large EST/mRNA dataset. The clustering is performed by a slightly modified version of NCBI's megablast , and the resulting clusters are then assembled using CAP3 assembly program. TGICL starts with a large multi-FASTA file (and an optional peer quality values file) and outputs the assembly files as produced by CAP3. Both clustering and assembly phases can be parallelized by distributing the searches and the assembly jobs across multiple CPUs, as TGICL can take advantage of either SMP machines or PVM (Parallel Virtual Machine) clusters. The two full precompiled packages below were built on Linux and SunOS, respectively. They include CAP3, mgblast and all the other binaries for this platform (of course, except the base Unix utilities like 'sed', 'sort' etc.). ===Availability=== Version 2.1 ===Use=== module add TGICL-2.1 tgicl see Documentation. Adding of module will copy configuration file /software/TGICL-2.1/build/etc/tgicl.cfg to the actual directory. This cfg file must be edited. ===Documentation=== http://compbio.dfci.harvard.edu/tgi/software/ ===Licence=== Artistic licence ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://compbio.dfci.harvard.edu/tgi/software/ [[Kategorie:Applications]] 57pmqdnroofg6ohi44s7wabjmxrofi8 0 256 5099 4860 2016-05-13T09:13:53Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--WRF|Česká verze--> == Description == The Weather Research and Forecasting (WRF) Model is a next-generation mesoscale numerical weather prediction system designed to serve both operational forecasting and atmospheric research needs. You can find more info on [http://wrf-model.org/index.php external pages]. == Availability == Currently there are two versions available on MetaCentrum machines: 3.3.1 and 3.4.1. == Licence == Public domain &ndash; freely available for all. == Use == Initialize module with command: module add wrf-3.4.1 After initialization are available all necessary system variables: $NETCDF, $COMPRESSION_LIBS, $WRFIO_NCD_LARGE_FILE_SUPPORT $JASPERINC, $JASPERLIB, $NCARG_ROOT, $LD_LIBRARY_PATH, $PATH The program needs working files (files with data and compute setup) in it's installation dir. The module overcomes this behavior by linking all the program files to the actual working directory. ===Usage examples=== Assuming that in <tt>/scratch/$USER/wrf_test</tt> directory is initialized the wrf-3.4.1 module and that the computing data are copied into <tt>/scratch/$USER/wrf_test/WRF-data</tt>. mkdir /scratch/$USER/wrf_test mkdir /scratch/$USER/wrf_test/WRF-data cp path_to_prepared_data /scratch/$USER/wrf_test/WRF-data Computing script shoud contain at the beginning defined variables and system variables: WPSDIR="/scratch/$USER/wrf_test/WPS" WRFDIR="/scratch/$USER/wrf_test/WRFV3" WRFDATA_UNGRIB="/scratch/$USER/wrf_test/WRF-data/metgrid/ds335.0/GFS_Global_0p5deg_2009010[1-2]*_anl.grib2" NAMELIST_WPS_FILE="/scratch/$USER/wrf_test/WRF-data/namelist.wps" NAMELIST_INPUT_FILE="$WRFDIR/test/em_real/namelist.input" export OMP_NUM_THREADS=$PBS_NUM_PPN Computation can be run the serial or parallel way: ====serial usecase==== # prepare and run the geogrid cd $WPSDIR ln -sf ungrib/Variable_Tables/Vtable.GFS Vtable [ ! $NAMELIST_WPS_FILE -ef namelist.wps ] && cp $NAMELIST_WPS_FILE . ./geogrid.exe 2>&1 | tee geogrid_log # prepare and run the ungrib ./link_grib.csh $WRFDATA_UNGRIB ./ungrib.exe 2>&1 | tee ungrib_log # run the metgrid ./metgrid.exe 2>&1 | tee metgrid_log # prepare and run the wrf cd $WRFDIR/test/em_real [ ! $NAMELIST_INPUT_FILE -ef namelist.input ] && cp $NAMELIST_INPUT_FILE . ln -sf $WPSDIR/met_em* . ./real.exe 2>&1 | tee real_log ./wrf.exe 2>&1 | tee wrf_log ====parallel usecase==== # prepare and run the geogrid cd $WPSDIR ln -sf ungrib/Variable_Tables/Vtable.GFS Vtable [ ! $NAMELIST_WPS_FILE -ef namelist.wps ] && cp $NAMELIST_WPS_FILE . mpirun ./geogrid.exe 2>&1 | tee geogrid_log # prepare and run the ungrib ./link_grib.csh $WRFDATA_UNGRIB ./ungrib.exe 2>&1 | tee ungrib_log #mpirun ./ungrib.exe 2>&1 | tee ungrib_log # the serial version performs slightly better :-/ # run the metgrid mpirun ./metgrid.exe 2>&1 | tee metgrid_log # prepare and run the wrf cd $WRFDIR/test/em_real [ ! $NAMELIST_INPUT_FILE -ef namelist.input ] && cp $NAMELIST_INPUT_FILE . ln -sf $WPSDIR/met_em* . mpirun ./real.exe 2>&1 | tee real_log mpirun ./wrf.exe 2>&1 | tee wrf_log == Documentation == http://www.mmm.ucar.edu/wrf/users/docs/user_guide/contents.html == Program Manager == [mailto:meta@cesnet.cz meta@cesnet.cz] == Homepage == http://wrf-model.org/index.php [[Category:Applications]] kezg43z009adit9lra88shrysxeipow 0 263 637 636 2016-01-23T18:50:07Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Příčiny nestandardního skončení úlohy|Česká verze]] This page describes the possible causes of unnatural job end by the scheduling system. The [[Scheduling system - detailed description|scheduling system]] kills the job in the situations when the job exceeds memory, processors or time length limits. Disrespect to [[#You forgot to set some limit|required limits]] leads always to unnatural job end due to an effort to protect other users from expansiveness of your job. To solve the actual problem once more read the [[Scheduling system - detailed description|scheduling system description]] and notes for [[:Category:Applications|your application]]. =What resources are supervised= * walltime &ndash; how long is the job running. It cannot run longer than the defined [[Scheduling system - detailed description#Available queues|queue time]] or than is specified in parameter ''walltime''. * used CPUs &ndash; the job is killed when it computed on more processors than were granted by the scheduling system. * used memory &ndash; the memory occupied by the running job. ** '''Note:''' Qsub recognizes two parameters regarding memory - ''mem'' and ''vmem''. For running you jobs please use only ''mem'' parameter, ''vmem'' (= mem + swap) is only for special purposes which, the most probably, you will not meet. The default of allocated resources when no qsub parameter is specified, is queue normal 1 node 1 processor 400MB of memory =Recommendations= ===You forgot to set some limit=== If you don't specify the queue and limit of memory, processors and nodes, the default stat of the job is queue <tt>normal</tt>, 400MB of memory, 1 node and 1 processor. More about qsub parameters can be found at [[Scheduling system - detailed description|dedicated page]]. You should always set the number of nodes, processors, memory limit and scratch size. ===Tuning up the resources needs on the test job=== For the first try to run one job of the specific type for the test and check it's needs. Other job of the same type can be run afterwards with tuned parameters. It is worth to know hom much memory and disc space the job needs before you run several jobs and system kills them all. You can see details of your jobs in the [http://metavo.metacentrum.cz/pbsmon2/person PBSmon] including it's specified and consumed resources. Unfortunately PBSmon shows the jobs quite complex so it does not show the consumed resources on each used node but in the whole. You can find more in section [[#Requests don't count with number of machines|Requests don't count with number of machines]]. At the beginning is good to demand more resources and then lower the needs. It can take longer to gather required resources for the test job, but the other jobs will be planned faster. You can see the computing needs of your job by planning the [[interactive job]] at the machine with sufficient resources and observe how many threads runs on what processors (<tt>htop</tt>). This can be done quite quickly, the job don't have to end, you can kill it after several seconds. The most frequent errors are in wrong run of parallel jobs and in wrong set up of certain programs (Java, Matlab). You can find more by reading the documentation for [[:Category:Applications|specific applications]]. The memory needs of the job can be found in [http://metavo.metacentrum.cz/pbsmon2/person PBSmon] after the end of the job because it can vary during the computation. ===Requests don't count with number of machines=== You have specified more nodes in <tt>qsub</tt> (ie. <tt>-l nodes=2:ppn=3 -l mem=4gb</tt>) but you forgot that memory is allocated for one machine. In other words for your job is reserved 8GB of memory and 6 processors, but on the 2 machines so the job can not exceed 4GB of memory and 3 processors at one machine. This principle is influenced by license needs of various SW, such as [[Gaussian-GaussView application#Availability|Gaussian]]. ===The most frequent wrongly used programs=== Some programs are more likely to exceed the requested resources regarding the memory limits. It is mainly [[Running the application on JAVA platform|Java]] and [[Matlab application|Matlab]]. The examples of correct usage of these programs are here in wiki. [[Category:English]] [[Category:Jobs and batch system]] [[Category:Applications]] [[Category:Help]] lyybtc2fy83n5m6ky0qkaz4us5f3x0q 0 264 5145 5129 2016-05-13T09:14:32Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Cluto|Česká verze--> ===Description=== '''CLUTO''' is a software package for clustering low- and high-dimensional datasets and for analyzing the characteristics of the various clusters. CLUTO is well-suited for clustering data sets arising in many diverse application areas including information retrieval, customer purchasing transactions, web, GIS, science, and biology. CLUTO's distribution consists of both stand-alone programs (<code>scluster</code> and <code>vcluster</code>) and a library (<code>libcluto.a</code>) via which an application program can access directly the various clustering and analysis algorithms implemented in CLUTO. A simple perl script called <code>doc2mat</code> that convertes a set of documents into the vector-space format used by CLUTO is also provided. ===Availability=== '''Module <code>cluto-2.1.2a</code>:''' latest stable release, includes the Perl script <code>doc2mat</code> ===Use=== <source lang="bash"> $ module add cluto-2.1.2a $ doc2mat [options] doc-file mat-file $ vcluster [optional parameters] MatrixFile NClusters $ scluster [optional parameters] GraphFile NClusters </source> ===Documentation=== * CLUTO docs: [http://glaros.dtc.umn.edu/gkhome/fetch/sw/cluto/manual.pdf provider's webpage] or locally within the program directory (<code>/software/cluto-2.1.2a/doc/manual.pdf</code>) * <code>doc2mat</code> docs: [http://glaros.dtc.umn.edu/gkhome/files/fs/sw/cluto/doc2mat.html provider's webpage] or locally within the program directory (<code>/software/cluto-2.1.2a/doc/doc2mat.html</code>) ===License=== Freely available for educational and research purposes by non-profit institution. ===Supported platforms=== amd64 ===Program administrator=== Tom Rebok [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://glaros.dtc.umn.edu/gkhome/cluto/cluto/overview [[Kategorie:Applications]] 2tyxo8do4nd8x1rqfwicsgewyhvfxb8 0 266 5061 4938 2016-05-13T09:13:17Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Bedtools|Česká verze--> =Description= Collectively, the bedtools utilities are a swiss-army knife of tools for a wide-range of genomics analysis tasks. The most widely-used tools enable genome arithmetic: that is, set theory on the genome. For example, bedtools allows one to intersect, merge, count, complement, and shuffle genomic intervals from multiple files in widely-used genomic file formats such as BAM, BED, GFF/GTF, VCF. =Availability= Version 2.17 =Licence= [http://www.gnu.org/licenses/old-licenses/gpl-2.0.html GNU GPL v2] =Use= You can load the module via the command module add bedtools-2.17 This makes available the path to the program binaries and loads also the [[SAMtools application|SAMtools]] module. =Documentation= http://bedtools.readthedocs.org/en/latest/index.html ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://bedtools.readthedocs.org/en/latest/index.html [[Category:Applications]] [[Category:English]] 085bd8nh1g07qka8h29nbdpf0ezsbvw 0 270 5054 4927 2016-05-13T09:13:10Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Bioperl|Česká verze--> =Description= BioPerl is a toolkit of perl modules useful in building bioinformatics solutions in Perl. Includes binaries and Perl modules useful in user scripts. =Availability= Version 1.6.1. Freely available to all users. For list of installed modules see the page [[Perl - modules]]. Available versions: bioperl -> bioperl-1.6.1 bioperl-1.6.9 =Licence= [http://www.bioperl.org/wiki/Perl_Artistic_License Perl Artistic License] =Use= You can load the module via the command module add bioperl This makes available the path to the program binaries, Perl libraries and manual pages. To use libraries in own scripts try this: #!/usr/bin/perl use Bio::Perl; own script code =Documentation= Documentation is available in manual pages or at the project [http://www.bioperl.org/wiki/Main_Page wiki]. ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.bioperl.org/wiki/Main_Page [[Category:Applications]] [[Category:English]] 7r3igq4pgiemqyg9cu0gay30ctamktt 0 276 5078 4798 2016-05-13T09:13:33Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Stacks|Česká verze--> = Description = Stacks is a software pipeline for building loci from short-read sequences, such as those generated on the Illumina platform. Stacks was developed to work with restriction enzyme-based data, such as RAD-seq, for the purpose of building genetic maps and conducting population genomics and phylogeography. = Availability = Version 1.02, 1.19, 1.34 and 1.35 = Licence = GNU GPL license = Use = module add stacks-1.34 process_radtags = Documentation = http://creskolab.uoregon.edu/stacks/stacks_manual.pdf = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://creskolab.uoregon.edu/stacks/ [[Category:Applications]] [[Category:English]] kuypfhr1ylgy4rluc3hdu6nem38o3mz 0 278 5155 4820 2016-05-13T09:14:41Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Symos'97|Česká verze--> ===Description=== '''Symos'97''' is ... ===Availability=== '''Module <code>symos97-6.0</code>:''' ''Symos'97 version 6.0'', 64-bit The available licence '''allows just the members of VŠB-TU in Ostrava''' to use the software. ===Use=== ... ===Documentation=== http://www.idea-envi.cz/symos.html ===Licence=== komerční ===Supported platforms=== amd64 ===Program administrator=== Tom Rebok [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.idea-envi.cz/produkty_symos.html [[Kategorie:Applications]] 9lq4u63kdyqpwn9f07jj7d6txa09hnv 14 283 1926 1925 2016-04-30T13:45:18Z Root 1 wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[How to compute/Interactive jobs|Interactive jobs]]|}} }} Applications in this category can be used with Graphical User Interface ('''GUI'''). For more information, see [[Remote desktop]] or [[X-Window]]. [[Category:Applications]] 15kqbb4hwn6443ted3w8dce4fgj8trw 0 284 4837 4753 2016-05-13T09:08:54Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Bfast|Česká verze--> =Description= BFAST facilitates the fast and accurate mapping of short reads to reference sequences. =Availability= Version 0.7.0 =Use= module add bfast-0.7.0 bfast or run other blast binaries (see /software/bfast-0.7.0/bin or documentation in file /software/bfast-0.7.0/share/doc/bfast/bfast-book.pdf). =Documentation= File /software/bfast-0.7.0/share/doc/bfast/bfast-book.pdf or =Licence= It was written as part of the author's official duties as a United States Government employee and thus cannot be copyrighted. This software/database is freely available to the public for use. The National Library of Medicine and the U.S. Government have not placed any restriction on its use or reproduction. =Program manager= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://sourceforge.net/apps/mediawiki/bfast/index.php?title=Main_Page [[Category:Applications]] [[Category:English]] e43m4yjy9rzfmkczgvn2afajdisfpwt 0 285 4974 4854 2016-05-13T09:11:47Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--QIIME|Česká verze--> = Description = QIIME (pronounced "chime") stands for Quantitative Insights Into Microbial Ecology. QIIME is an open source software package for comparison and analysis of microbial communities, primarily based on high-throughput amplicon sequencing data (such as SSU rRNA) generated on a variety of platforms, but also supporting analysis of other types of data (such as shotgun metagenomic data). QIIME takes users from their raw sequencing output through initial analyses such as OTU picking, taxonomic assignment, and construction of phylogenetic trees from representative sequences of OTUs, and through downstream statistical analysis, visualization, and production of publication-quality graphics. QIIME has been applied to single studies based on billions of sequences from thousands of samples. Installed tools (see http://qiime.org/ ) * biom-format * blast * [[Cdbfasta application|Cdbfasta]] * clearcut * fasttree * matplotlib * PyNAST * pyqi * raxml * RDP * rtax * tax2tree * uclust * usearch Greengenes files are in /software/qiime-1.7.0 = Availability = Version 1.7.0 = Licence = Depends on used tool. = Use = module add qiime-1.7.0 align_seqs.py -h = Documentation = http://qiime.org/documentation_index.html = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://qiime.org/ [[Category:Applications]] [[Category:English]] ni3rkwa82m24g1wsy7zmgptmlhgfpha 0 289 5194 4918 2016-05-13T09:15:14Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Minia|Česká verze--> = Description = Minia is a short-read assembler based on a de Bruijn graph, capable of assembling a human genome on a desktop computer in a day. The output of Minia is a set of contigs. Minia produces results of similar contiguity and accuracy to other de Bruijn assemblers (e.g. Velvet). = Availability = Version 1.5418 (compiled with k=100) = Licence = Not specified. = Use = module add minia-1.5418 ./minia = Documentation = http://minia.genouest.org/ = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://minia.genouest.org/ [[Category:Applications]] [[Category:English]] pocizgi60in61y1rh79qcyo2a67nrz6 0 291 5035 4910 2016-05-13T09:12:54Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Last|Česká verze--> = Description = *LAST finds similar regions between sequences. LAST can: *Handle big sequence data, e.g: **Compare two vertebrate genomes **Align billions of DNA reads to a genome *Indicate the reliability of each aligned column. *Use sequence quality data properly. *Compare DNA to proteins, with frameshifts. *Compare PSSMs to sequences *Calculate the likelihood of chance similarities between random sequences. = Availability = Version 621(intel + gcc compilers) = Licence = Not specified. = Use = module add last-621 lastdb = Documentation = http://last.cbrc.jp/ = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://last.cbrc.jp/ [[Category:Applications]] [[Category:English]] tveswieq9o9z81d0kpvs5ondbn8gz89 0 292 5088 4805 2016-05-13T09:13:40Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--VisIt|Česká verze--> = Description = '''VisIt''' is a free interactive parallel visualization and graphical analysis tool for viewing scientific data on Unix and PC platforms. Users can quickly generate visualizations from their data, animate them through time, manipulate them, and save the resulting images for presentations. VisIt contains a rich set of visualization features so that you can view your data in a variety of ways. It can be used to visualize scalar and vector fields defined on two- and three-dimensional (2D and 3D) structured and unstructured meshes. VisIt was designed to handle very large data set sizes in the terascale range and yet can also handle small data sets in the kilobyte range. = Availability = '''VisIt version 2.6.3''': module <code>visit-2.6.3</code> = Use = Load the relevant module and run the VisIt. The application requires graphical interface. <source lang=bash> $ module add visit-2.6.3 $ visit </source> {{Application with GUI}} = Documentation = The VisIt documentation is available at [https://wci.llnl.gov/codes/visit//manuals.html official webpages]. = Program administrator = * Martin Jirka -- KFE FJFI, Czech Technical University in Prague (jirkama1 at fjfi.cvut.cz) * Tomáš Rebok (rebok at ics.muni.cz) = Homepage = URL: https://wci.llnl.gov/codes/visit//home.html [[Category:Applications]] [[Category:English]] nwcdkwuqamtnj1y6tr9qy9uu1gcpuoy 0 298 4925 4913 2016-05-13T09:10:52Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--CLCbio_Genomics_Workbench|Česká verze--> = Description = The '''CLCbio Genomics Workbench''', for analyzing and visualizing next generation sequencing data, incorporates cutting-edge technology and algorithms, while also supporting and integrating with the rest of your typical NGS workflow. It enables everyone to rapidly analyze and visualize the huge amounts of data generated by NGS machines. The user-friendly and intuitive interface essentially takes High-Throughput Analysis away from hardcore bioinformatics programmers doing command-line scripts, and hands it to scientists searching for biological results. Furthermore, the versatile nature of CLC Genomics Workbench allows it to blend seamlessly into existing sequencing analysis workflows, easing an implementation. = Availability = '''''CLCbio Genomics Workbench version 7.0''''': module <code>CLCgenomicsWB-7.0</code> * upgrade of a license purchased by '''Centre CERIT-SC''' (available for all the MetaVO users) allowing '''''2 simultaneous runs of the application''''' * equipped with a set of available extensions '''CLCbio Genomics Workbench version 6.5.1''': module <code>CLCgenomicsWB-6.5</code> * a license purchased by '''Centre CERIT-SC''' (available for all the MetaVO users) allowing '''''2 simultaneous runs of the application''''' * equipped with a set of available extensions = Use = {{Rámeček-green | text='''''Important:''' When requesting a job that will use the application, do not forget to '''ask the scheduler for starting the job when an application license is available''', i.e., submit the job with e.g.'' <code>qsub -l nodes=1:ppn=X:... -l clcgenomics=1</code> ''when requesting a single license to be available.''}} Load the appropriate module and run the CLCbio Genomics Workbench application. The application requires a graphical interface available. <source lang=bash> $ module add CLCgenomicsWB-7.0 $ clcgenomicswb7 </source> {{Application with GUI}} = Documentation = Documentation is available on [http://www.clcsupport.com/clcgenomicsworkbench/current/index.php?manual=Introduction_CLC_Genomics_Workbench.html the official producer's webpage] or locally in the program directory (<code>/software/clcgenomicswb/v7.0/doc/</code> or <code>/software/clcgenomicswb/v6.5/doc/</code>) = Program administrator = * Tomáš Rebok (meta at cesnet.cz) = Homepage = URL: http://www.clcbio.com/ [[Category:Applications]] [[Category:English]] 7y7pbbokfiy997xqe18tvfvshnalovh 0 299 5112 5107 2016-05-13T09:14:03Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Geneious|Česká verze--> = Description = The '''Geneious''' is an integrated, cross-platform bioinformatics software suite developed by Biomatters for manipulating, finding, sharing, and exploring biological data such as DNA sequences or proteins, phylogenies, 3D structure information and publications. It features sequence alignment and phylogenetic analysis, contig assembly, primer design and restriction analysis, access to the NCBI and UniProt databases, BLAST, protein structure viewing, automated PubMed searching, and more. One of the main advantages is its strong focus on user-friendly interface and ease of use. = Availability = '''Geneious release R7 (version 7.1.5)''': module <code>geneious-7.1.5</code> * Geneious release R7 (version 7.0.3): module <code>geneious-R7</code> * a license purchased by '''CERIT-SC Centre''' allowing '''''2 simultaneous runs''''' (available for all the MetaVO users) = Use = {{Rámeček-green | text='''''Important:''' When requesting a job that will use the application, do not forget to '''ask the scheduler for starting the job when an application license is available''', i.e., submit the job with e.g.'' <code>qsub -l nodes=1:ppn=X:... -l geneious=1</code> ''when requesting a single license to be available.''}} Load the relevant module and run the Geneious application. The application requires a graphical interface. <source lang=bash> $ module add geneious-7.1.5 $ Geneious </source> {{Application with GUI}} = Documentation = The Geneious documentation is available on the [http://assets.geneious.com/documentation/geneious/GeneiousManual.pdf official producer's webpage] = Program administrator = * Tomáš Rebok (meta at cesnet.cz) = Homepage = URL: http://www.geneious.com [[Category:Applications]] [[Category:English]] tgzmch57zb916b4iie7izasaanwdijn 0 301 5156 5071 2016-05-13T09:14:41Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--FastQC|Česká verze--> ===Description=== '''FastQC''' is ... ===Availability=== '''Module <code>fastQC-0.10.1</code>:''' ''FastQC version 0.10.1'', 64-bit ===Use=== ===Documentation=== ===License=== ===Supported platforms=== amd64 ===Program administrator=== Petr Navrátil, Tom Rebok [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== [[Kategorie:Applications]] [[Category:Applications]] mkz99qaahlms4v050exu9vpz9wfp041 0 304 5075 5067 2016-05-13T09:13:30Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--FreeBayes|Česká verze--> = Description = '''FreeBayes''' is a Bayesian genetic variant detector designed to find small polymorphisms, specifically SNPs (single-nucleotide polymorphisms), indels (insertions and deletions), MNPs (multi-nucleotide polymorphisms), and complex events (composite insertion and substitution events) smaller than the length of a short-read sequencing alignment. It uses short-read alignments (BAM files with Phred+33 encoded quality scores) for any number of individuals from a population and a reference genome to determine the most-likely combination of genotypes for the population at each position in a reference genome (FASTA). It reports positions which it finds to be more likely polymorphic than monomorphic in a standard variant interchange format (VCF). It can also use an input set of variants (VCF) as a source of prior information, and a copy number variant map (BED) to define non-uniform ploidy variation across the samples under analysis. = Availability = '''Module''' freebayes-9.9.2 = Licence = Copyright (c) 2010 Erik Garrison, Gabor Marth Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. = Use = freebayes [OPTION] ... [BAM FILE] ... = Documentation = See [http://arxiv.org/abs/1207.3907] for an overview of the statistical models used in FreeBayes. We ask that you cite this paper if you use FreeBayes in work that leads to publication. = Program manager = Štěpán Stočes (UEB AS CR), Tom Rebok ([mailto:meta@cesnet.cz meta@cesnet.cz]) = Homepage = https://github.com/ekg/freebayes [[Category:Applications]] [[Category:English]] k1jxrdkllz3huil4ex21mc53pfbzodp 0 307 5186 5056 2016-05-13T09:15:08Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--ATSAS|Česká verze--> =Description= A program suite for small-angle scattering data analysis from biological macromolecules. =Availability= Version 2.5.1. Freely available for academic and education usage. =Licence= Own licence, free for academic and education usage. See <tt>/software/atsas-2.5.1/share/doc/atsas-2.5.1/LICENSE.txt</tt>. =Use= You can load the module via the command module add atsas-2.5.1 This makes available the path to the program binaries. =Documentation= Documentation for ATSAS commands is in directory <tt>/software/atsas-2.5.1/share/doc/atsas-2.5.1/</tt> or at pages http://www.embl-hamburg.de/biosaxs/manuals/. ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.embl-hamburg.de/biosaxs/software.html [[Category:Applications]] [[Category:English]] 74dj2st0q4xodkwf53pxzkbvrj9vixv 0 309 5089 4961 2016-05-13T09:13:41Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--HEAsoft|Česká verze--> = Description = '''HEAsoft''' je souhrnné označení pro soubor softwaru pro správu a analýzu dat vetšinou družicových experimentů v oboru astrofyziky vysokých energií. Obsahuje mimo základní sadu nástrojů pro manipulaci se soubory ve FITS formátu (FTOOLS) také balíky '''XSPEC''' (analýza spekter), XRONOS (časové řady) a specifické nástroje pro jednotlivé mise (experimenty). = Availability = Verze 6.14 dostupná každému uživateli. Zdarma ke stažení [[http://heasarc.gsfc.nasa.gov/docs/software/lheasoft/download.html]] (vzhledem k limitované rychlosti stahování může přenos z GSFC trvat velmi dlouho). = Licence = Zdrojový kód je dostupný k volnému šíření pod GPL. = Use = Je třeba zadefinovat proměnnou export HEADAS='/software/heasoft-6.15/x86_64-unknown-linux-gnu-libc2.5' a poté spustit inicializační skript . $HEADAS/headas-init.sh (analogicky lze provést inicializaci v C-shellu pomocí $HEADAS/headas-init.csh) = Documentation = = Program manager = Filip Münz, Matúš Kocka, Tomáš Rebok ([mailto:meta@cesnet.cz meta@cesnet.cz]) = Homepage = [http://heasarc.nasa.gov/lheasoft/] On-line manuál pro XSPEC najdete zde [http://heasarc.gsfc.nasa.gov/docs/software/lheasoft/xanadu/xspec/manual/manual.html] [[Category:Applications]] [[Category:English]] 57s4vpr0z9kz52sx7p24sq03v8wlwsu 0 310 4942 4780 2016-05-13T09:11:09Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--CIAO|Česká verze--> = Description = '''CIAO''' is ... = Availability = = Licence = = Use = = Documentation = = Program manager = Filip Münz, Matúš Kocka, Tomáš Rebok ([mailto:meta@cesnet.cz meta@cesnet.cz]) = Homepage = http://cxc.harvard.edu/ciao/ [[Category:Applications]] [[Category:English]] ko8ct18rohobhc5o7jldzm790t1jlry 0 312 5143 4914 2016-05-13T09:14:30Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Fermi_Science_Tools|Česká verze--> = Description = '''Fermi Science Tools''' is ... = Availability = = Licence = = Use = = Documentation = = Program manager = Filip Münz, Matúš Kocka, Tomáš Rebok ([mailto:meta@cesnet.cz meta@cesnet.cz]) = Homepage = http://fermi.gsfc.nasa.gov/ssc/ [[Category:Applications]] [[Category:English]] my00etn70b0zpp64vks7r2otj88eoom 0 314 4964 4945 2016-05-13T09:11:31Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--FASTQ_Illumina_Filter|Česká verze--> ===Description=== Illumina CASAVA-1.8 FASTQ Filter is a program that can filter FASTQ files produced by CASAVA 1.8, and keep/discard reads based on this filter flag. ===Use=== Load module (module add fastq_illumina_filter-0.1) and run by calling "fastq_illumina_filter". ===License=== AGPL v3 ===Supported platforms=== Linux ===Module administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://cancan.cshl.edu/labmembers/gordon/fastq_illumina_filter/ [[Kategorie:Applications]] kvvezcmq65jrgkypfi76bt6erefpvhp 14 316 1927 1515 2016-04-30T13:50:33Z Root 1 wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[How to compute/Parallel jobs|Parallel jobs]]|}} }} This is an automatically generated list of all applications supporting MPI. You can read more about parallel computing [[How to compute/Parallel jobs|here]]. [[Category:Applications]] r3rh35p646fbroe77lo9p6v0jaiqtvc 0 319 4950 4935 2016-05-13T09:11:16Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Masurca|English version--> = Description = MaSuRCA is whole genome assembly software. It combines the efficiency of the de Bruijn graph and Overlap-Layout-Consensus (OLC) approaches. MaSuRCA can assemble data sets containing only short reads from Illumina sequencing or a mixture of short reads and long reads (Sanger, 454). = Availability = Version 2.1.0. Freely available to users. Modules: masurca-2.1.0 = Licence = [http://www.gnu.org/copyleft/gpl.html GPLv3] ? = Use = Initialize environment with command: module add masurca-2.1.0 = Documentation = Documentation is in file ftp://ftp.genome.umd.edu/pub/MaSuRCA/MaSuRCA_QuickStartGuide.pdf or locally in <tt>/software/masurca-2.1.0/doc/MaSuRCA_QuickStartGuide.pdf</tt>. = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://genome.umd.edu/masurca.html [[Category:Applications]] [[Category:English]] h9avuxzlls1d2h9gr4hph2z26vumg2m 0 322 4908 4845 2016-05-13T09:10:22Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Orca|Česká verze--> = Description = ORCA is a flexible, efficient and easy-to-use general purpose tool for quantum chemistry with specific emphasis on spectroscopic properties of open-shell molecules. It features a wide variety of standard quantum chemical methods ranging from semiempirical methods to DFT to single- and multireference correlated ab initio methods. It can also treat environmental and relativistic effects. Due to the user-friendly style, ORCA is considered to be a helpful tool not only for computational chemists, but also for chemists, physicists and biologists that are interested in developing the full information content of their experimental data with help of calculations. = Availability = Version 2.6.4 and 3.0.1. Available for every academic user after registration. Modules: orca-2.6.4 orca-3.0.1 {{Odsouhlasení licence (en)}} = Licence = http://cec.mpg.de/forum/license.html = Use = Example of environment initialization: module add orca-3.0.1 {{MPI application}} = Documentation = Dokumentation is available at http://cec.mpg.de/forum/OrcaManual.pdf or in file <tt>/software/orca-3.0.1/doc/OrcaManual.pdf</tt>. = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://cec.mpg.de/forum/ [[Category:Applications]] [[Category:English]] 0sf8i0l5w03hli4ll3k5z0wpb5y028f 0 324 4858 4774 2016-05-13T09:09:25Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--CTHYB|Česká verze--> = Description = = Availability = = Licence = GPL, version 3 = Use = module add cthyb pytriqs my.py = Documentation = http://ipht.cea.fr/triqs/applications/cthyb_matrix/documentation.html = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://ipht.cea.fr/triqs/applications/cthyb_matrix/index.html [[Category:Applications]] [[Category:English]] oersdww5nrgieklmmqmjecv7v9xdv6l 0 330 5210 5039 2016-05-13T09:15:34Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--CD-HIT|Česká verze--> = Description = CD-HIT is a very widely used program for clustering and comparing protein or nucleotide sequences. It is very fast and can handle extremely large databases. CD-HIT helps to significantly reduce the computational and manual efforts in many sequence analysis tasks and aids in understanding the data structure and correct the bias within a dataset. = Availability = Version 4.6.1. Freely available to users. Modules: cdhit-4.6.1 = Licence = [http://www.gnu.org/licenses/gpl-2.0.html GPLv2] = Use = Initialize environment with command: module add cdhit-4.6.1 = Documentation = Documentation is at http://weizhong-lab.ucsd.edu/cd-hit/ref.php . = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://weizhong-lab.ucsd.edu/cd-hit/ [[Category:Applications]] [[Category:English]] fmbopd5lan1e0hgr35vid788rsfr8ku 0 331 5013 4923 2016-05-13T09:12:32Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--EddyPro (Smartflux)|Česká verze--> = Description = EddyPro is software for processing raw eddy covariance (EC) data to compute biosphere/atmosphere fluxes of CO₂, H₂O, CH₄, other trace gases, and energy. It is based on the ECO₂S package, which was released as free software by the consortium participating in the IMECC-EU European project. EddyPro is customized to efficiently process eddy covariance data logged to LI‑COR® gas analyzers (.ghg files). It also supports other raw file types, including data stored as ASCII tables, binary files (including the TOB1 format), and SLT (EddySoft and EdiSol) formats. = Availability = Version 1.2.0, 5.2 and 6.0. Freely available to users. Modules: smartflux-1.2.0 eddypro-20140509 eddypro-5.2 eddypro-6.0 (newest) = Licence = [http://www.gnu.org/copyleft/gpl.html GPLv3]? = Use = Initialize environment with command: module add eddypro-5.2 module add eddypro-6.0 = Documentation = Documentation is at http://envsupport.licor.com/help/EddyPro5/index.htm = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://www.licor.com/env/products/eddy_covariance/smartflux.html [[Category:Applications]] [[Category:English]] ertp16isrqvxneiiaw1lz0l4j6bubr3 0 339 5154 4859 2016-05-13T09:14:40Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--SGA|Česká verze--> = Description = SGA is a de novo genome assembler based on the concept of string graphs. The major goal of SGA is to be very memory efficient, which is achieved by using a compressed representation of DNA sequence reads. = Availability = Version 0.10.13. Freely available for every user. = Licence = [http://www.gnu.org/copyleft/gpl.html GNU/GPL v3] = Use = Initialize module with command: module add sga-0.10.13 = Documentation = Documentation is available at [https://github.com/jts/sga/wiki SGA wiki], first steps are in [https://github.com/jts/sga/tree/master/src#readme README] file. = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = https://github.com/jts/sga [[Category:Applications]] [[Category:English]] t9l5jvzb9gbc7tfe3yjzw1e4wdi1jz9 0 340 4928 4784 2016-05-13T09:10:54Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Quake|Česká verze--> = Description = Quake is a package to correct substitution sequencing errors in experiments with deep coverage (e.g. >15X), specifically intended for Illumina sequencing reads. Quake adopts the k-mer error correction framework, first introduced by the EULER genome assembly package. Unlike EULER and similar progams, Quake utilizes a robust mixture model of erroneous and genuine k-mer distributions to determine where errors are located. Then Quake uses read quality values and learns the nucleotide to nucleotide error rates to determine what types of errors are most likely. This leads to more corrections and greater accuracy, especially with respect to avoiding mis-corrections, which create false sequence unsimilar to anything in the original genome sequence from which the read was taken. = Availability = Version 0.3.5. Freely available for every user. = Licence = [http://www.bioperl.org/wiki/Perl_Artistic_License Perl Artistic License] = Use = Initialize module with command: module add quake-0.3.5 = Documentation = Documentation is available at http://www.cbcb.umd.edu/software/quake/manual.html. = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://www.cbcb.umd.edu/software/quake/index.html [[Category:Applications]] [[Category:English]] rfjbdda6e19jet70la17exv1bebi7ag 0 343 4846 4829 2016-05-13T09:09:09Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--bayArea|Česká verze--> = Description = BayArea infers the biogeographic histories of taxa sharing a phylogeny, using Markov chain Monte Carlo (MCMC) to approximate the joint posterior of biogeographic histories and range evolution parameters given presence-absence (binary) data and a time-calibrated phylogeny. Model details are available in our forth- coming paper. BayArea is written in C++ and is open source. = Availability = Version 1.0.2 Freely available for every user. = Licence = [http://www.gnu.org/licenses/gpl.html GENERAL PUBLIC LICENSE] = Use = Initialize module with command: module add bayarea-1.0.2 = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = https://code.google.com/p/bayarea/ [[Category:Applications]] [[Category:English]] 2c9iajs5yjkyick6l7s8f97qz9w1gpa 0 344 5020 4917 2016-05-13T09:12:39Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--bayArea|Česká verze--> = Description = BayArea infers the biogeographic histories of taxa sharing a phylogeny, using Markov chain Monte Carlo (MCMC) to approximate the joint posterior of biogeographic histories and range evolution parameters given presence-absence (binary) data and a time-calibrated phylogeny. Model details are available in our forth- coming paper. BayArea is written in C++ and is open source. = Availability = Version 1.0.2 Freely available for every user. = Licence = [http://www.gnu.org/licenses/gpl.html GENERAL PUBLIC LICENSE] = Use = Initialize module with command: module add bayarea-1.0.2 = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = https://code.google.com/p/bayarea/ [[Category:Applications]] [[Category:English]] 2c9iajs5yjkyick6l7s8f97qz9w1gpa 0 347 4898 4848 2016-05-13T09:10:09Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--RSEM|Česká verze--> = Description = RSEM is a software package for estimating gene and isoform expression levels from RNA-Seq data. The RSEM package provides an user-friendly interface, supports threads for parallel computation of the EM algorithm, single-end and paired-end read data, quality scores, variable-length reads and RSPD estimation. In addition, it provides posterior mean and 95% credibility interval estimates for expression levels. For visualization, It can generate BAM and Wiggle files in both transcript-coordinate and genomic-coordinate. Genomic-coordinate files can be visualized by both UCSC Genome browser and Broad Institute’s Integrative Genomics Viewer (IGV). Transcript-coordinate files can be visualized by IGV. RSEM also has its own scripts to generate transcript read depth plots in pdf format. The unique feature of RSEM is, the read depth plots can be stacked, with read depth contributed to unique reads shown in black and contributed to multi-reads shown in red. In addition, models learned from data can also be visualized. Last but not least, RSEM contains a simulator. = Availability = Versions 1.2.8 and 1.2.20 = Licence = GPL version 3. = Use = module add resem-1.2.20 rsem-* = Documentation = http://deweylab.biostat.wisc.edu/rsem/ = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://deweylab.biostat.wisc.edu/rsem/ [[Category:Applications]] [[Category:English]] 7c8gqm027xblwofcd7aibt94idxsp9f 0 348 5171 4852 2016-05-13T09:14:55Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Tassel|Česká verze--> =Description= TASSEL has multiple functions, including association study, evaluating evolutionary relationships, analysis of linkage disequilibrium, principal component analysis, cluster analysis, missing data imputation and data visualization =Availability= Program is available in version 3.0 =Use= Initialize enviroment module add tassel-3.0 and run command ''tassel''. {{Application with GUI}} =License= [http://www.gnu.org/copyleft/gpl.html GNU GPL] =Documentation= Documentation is available in [http://www.maizegenetics.net/tassel/docs/Tassel_User_Guide_3.0.pdf pdf]. =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= URL: http://www.maizegenetics.net/index.php?option=com_content&task=view&id=89&Itemid=119 [[Category:Applications]] [[Category:English]] bbxp4wrt78x3eulkgwtvpqtqcf5xxsi 0 349 4801 4776 2016-05-13T09:08:06Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Apache Ant|Česká verze--> = Description = Apache Ant is a Java library and command-line tool whose mission is to drive processes described in build files as targets and extension points dependent upon each other. The main known usage of Ant is the build of Java applications. Ant supplies a number of built-in tasks allowing to compile, assemble, test and run Java applications. Ant can also be used effectively to build non Java applications, for instance C or C++ applications. More generally, Ant can be used to pilot any type of process which can be described in terms of targets and tasks. = Availability = Version 1.6 a 1.9. Freely available to all users. Modules: ant-1.6 ant-1.9 = Licence = [http://www.apache.org/licenses/LICENSE-2.0.html Apache v2] = Use = Module load example: module add ant-1.9 = Documentation = Documentation is available at http://ant.apache.org/manual/index.html. = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://ant.apache.org/index.html [[Category:Applications]] [[Category:English]] jzqpr9ip26wagex9ebbkp5fhws8c3gb 0 350 5191 4953 2016-05-13T09:15:12Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--SOAPaligner|Česká verze--> =Description= It is an updated version of SOAP software for short oligonucleotide alignment. The new program features in super fast and accurate alignment for huge amounts of short reads generated by Illumina/Solexa Genome Analyzer. Compared to soap v1, it is one order of magnitude faster. It require only 2 minutes aligning one million single-end reads onto the human reference genome. Another remarkable improvement of SOAPaligner is that it now supports a wide range of the read length. =Availability= Verze 2.21 =Use= module add soapalign-2.21 for more details see Documentation. =Documentation= http://soap.genomics.org.cn/soapaligner.html =Licence= GNU Public License (GPL) =Program manager= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://soap.genomics.org.cn/soapaligner.html [[Category:Applications]] [[Category:English]] 4fp2m3dvbg4rs2x4k9twh2hqxheg8mv 0 353 5051 4842 2016-05-13T09:13:09Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--GERP|Česká verze--> =Description= GERP - program for identification identifikaci conserved element base on zalozeny na quantification =Availability= Program is available in version from 22 May 2011 =Use= Initialize enviroment module add gerp-05-2011 gerpcol gerpelem =License= [http://www.gnu.org/copyleft/gpl.html GNU GPL] =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= URL: hhttp://mendel.stanford.edu/SidowLab/downloads/gerp/index.html [[Category:Applications]] [[Category:English]] j5qkj6zhswpzi0tl33t7s3tg7so7s3b 0 354 4819 4807 2016-05-13T09:08:38Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--DIYABC|Česká verze--> = Description = Do It Yourself ABC is a user-friendly approach to Approximate Bayesian Computation for inference on population history using molecular markers. = Availability = Version 2.0.4. Freely available for every user. Available modules: diyabc-2.0.4 = Licence = ? = Use = Example of module initialization: module add diyabc-2.0.4 diyabc --help = Documentation = http://www1.montpellier.inra.fr/CBGP/diyabc/download/file.php?id=210 or local copy in file <tt>/software/diyabc-2.0.4/doc/Notice_DIYABC_v2.0.4.pdf</tt>. = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = * http://www1.montpellier.inra.fr/CBGP/diyabc/index.php * [http://www1.montpellier.inra.fr/CBGP/diyabc/viewtopic.php?f=26&t=11569 program sources] [[Category:Applications]] [[Category:English]] p5bs0oocaa4dr39wa9p4vd632lxtyi6 0 357 5172 4946 2016-05-13T09:14:55Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--eXpress|Česká verze--> =Description= eXpress is a streaming tool for quantifying the abundances of a set of target sequences from sampled subsequences. =Availability= Version 1.5.1 =Use= module add express-1.5.1 for more details see Documentation. =Documentation= http://bio.math.berkeley.edu/eXpress/manual.html =Licence= Artistic License 2.0 =Program manager= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://bio.math.berkeley.edu/eXpress/index.html [[Category:Applications]] [[Category:English]] jzvmqarkry6q2cgp3j0mcnxl5ono92r 0 358 4999 4879 2016-05-13T09:12:13Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Cdbfasta|Česká verze--> =Description= fast indexing and retrieval of fasta records from flat file databases. =Availability= Version 0.99. Freely available for all users. =Licence= [http://www.gnu.org/licenses/old-licenses/gpl-2.0.html GNU/GPL v2 licence] =Use= module add cdbfasta After the module load all paths to necessary binaries are available. =Documentation= Documentation for available commands are available in manual pages. More detailed program documentation is at http://cdbfasta.sourceforge.net/. =Program manager= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://cdbfasta.sourceforge.net/ [[Category:Applications]] [[Category:English]] 4kzvjx2tf92nctnjigmerj8a5azq0mj 0 359 5115 4830 2016-05-13T09:14:05Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--MIReNA|Česká verze--> =Description= A tool to find microRNAs with high accuracy and no learning at genome scale and from deep sequencing data. =Availability= Version 2.0 =Use= module add mirena-2.0 for more details see Documentation. =Documentation= http://www.lgm.upmc.fr/mirena/index.html =Licence= CeCILL licence =Program manager= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://www.lgm.upmc.fr/mirena/index.html [[Category:Applications]] [[Category:English]] dk5kipqlqkbwge9gf6ldcjythqcxjmq 0 361 5087 4980 2016-05-13T09:13:40Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--SeqTools|Česká verze--> = Description = A suite of tools for visualising sequence alignments. Blixem is an interactive browser of pairwise alignments that have been stacked up in a "master-slave" multiple alignment; it is not a 'true' multiple alignment but a 'one-to-many' alignment. It displays an overview section showing the positions of genes and alignments around the alignment window, and a detail section showing the actual alignment of protein or nucleotide sequences to the genomic DNA sequence. Dotter is a graphical dot-matrix program for detailed comparison of two sequences. Every residue in one sequence is compared to every residue in the other, with one sequence plotted on the x-axis and the other on the y-axis. Noise is filtered out so that alignments appear as diagonal lines. Belvu is a multiple sequence alignment viewer and phylogenetic tool. It has an extensive set of user-configurable modes to color residues by conservation or by residue type, and some basic alignment editing capabilities. It can generate distance matrices between sequences and construct distance-based trees, either graphically or as part of a phylogenetic software pipeline. = Availability = Version 4.30 = Licence = GNU General Public License = Use = module add seqtools-4.30 belvu blixem dotter = Documentation = https://www.sanger.ac.uk/resources/software/seqtools/#t_doc = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = https://www.sanger.ac.uk/resources/software/seqtools/ [[Category:Applications]] [[Category:English]] luj7r9r69kdcge0yfu7v90d13yglhid 0 362 5012 4936 2016-05-13T09:12:31Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--LASTZ|Česká verze--> = Description = Program for aligning DNA sequences, a pairwise aligner. = Availability = Version 1.03.02 = Licence = unknown = Use = module add lastz-1.03.02 lastz (uses integer scores) lastz_D (uses floating-point scores) = Documentation = http://www.bx.psu.edu/miller_lab/dist/README.lastz-1.02.00/README.lastz-1.02.00a.html = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://www.bx.psu.edu/~rsharris/lastz/ [[Category:Applications]] [[Category:English]] 41v6l27t9b8sxp1n1cbfil79fd3n2mj 0 364 5038 4986 2016-05-13T09:12:57Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Phonopy|Česká verze--> =Description= Phonopy is an open source package of phonon calculations based on the supercell approach. =Availability= Program is available in version 1.8.2 a 1.9.1.3. Usable modules: phonopy-1.9.1-intel phonopy-1.9.1-gcc phonopy-1.8.2 =Use= Initialize environment with command ''module add''. Example: module add phonopy-1.9.1-gcc and run command ''phonopy''. =Documentation= Documentation is available on [http://megasun.bch.umontreal.ca/People/lartillot/www/index.htm WWW server]. =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= URL: http://megasun.bch.umontreal.ca/People/lartillot/www/index.htm [[Category:Applications]] [[Category:English]] mf3ppqnu5cmq000itkymt7tuj92wyb6 0 369 5141 4877 2016-05-13T09:14:27Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--BUCKy|Česká verze--> =Description= BUCKy is a free program to combine molecular data from multiple loci =Availability= Version 1.4.3 =Use= module add bucky =Licence= GNU GPL v2 licence =Program manager= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://www.stat.wisc.edu/~ane/bucky/index.html [[Category:Applications]] [[Category:English]] d33vryb6mmeeprtrhr3hjvq79n1gols 0 370 4981 4775 2016-05-13T09:11:52Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--BUCKy|Česká verze--> =Description= BUCKy is a free program to combine molecular data from multiple loci =Availability= Version 1.4.3 =Use= module add bucky-1.4.3 =Licence= GNU GPL v2 licence =Program manager= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://www.stat.wisc.edu/~ane/bucky/index.html [[Category:Applications]] [[Category:English]] e6vtfeaq7vhi06ljn2gtes4b987gas5 0 371 5177 5123 2016-05-13T09:15:02Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--SOAP|Česká verze--> =Description= SOAP is a program for efficient gapped and ungapped alignment of short oligonucleotides onto reference sequences. The program is designed to handle the huge amounts of short reads generated by parallel sequencing using the new generation Illumina-Solexa sequencing technology. SOAP is compatible with numerous applications, including single-read or pair-end resequencing, small RNA discovery and mRNA tag sequence mapping. SOAP is a command-driven program, which supports multi-threaded parallel computing, and has a batch module for multiple query sets. =Availability= Version 1.11 =Use= module add soap-1.11 =Licence= GNU GPL v2 licence =Program manager= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://soap.genomics.org.cn [[Category:Applications]] [[Category:English]] rkqkpuancdwbrou96bmp97yawo1ut20 0 372 5175 4920 2016-05-13T09:15:01Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--RAxML|Česká verze--> = Description = RAxML (Randomized Axelerated Maximum Likelihood) is a popular program for phylogenetic analyses of large datasets under maximum likelihood. = Availability = Version 8.1.17 (approx from 23.2.2015) = Licence = [http://www.gnu.org/copyleft/gpl.html GPL v3] = Use = Add module module add raxml-8.1.17 now you can use single processor, thread or MPI version by issue command raxmlHPC raxmlHPC-PTHREADS mpirun raxmlHPC-MPI Addition of module recognizes processor architecture (if it supports AVX or not). {{MPI application}} = Documentation = https://github.com/stamatak/standard-RAxML = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = https://github.com/stamatak/standard-RAxML [[Category:Applications]] [[Category:English]] d1xyv92s0aiv6yz84l3imtnvoov6mrm 0 378 5216 5006 2016-05-13T09:15:40Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--PhyBin|Česká verze--> ===Description=== PhyBin is a simple command line tool that classifies a set of Newick tree files by their topology. The purpose of it is to take a large set of tree files and browse through the most common tree topologies. ===Availability=== phybin-0.3 ===Use=== module add phybin-0.3 phybin ===Documentation=== http://www.cs.indiana.edu/~rrnewton/projects/phybin/#downloading-and-install ===Licence=== ??? ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.cs.indiana.edu/~rrnewton/projects/phybin/#downloading-and-install [[Kategorie:Applications]] 4ud7vxpdg71cfmovj4dwn689zr87kmt 0 380 4970 4889 2016-05-13T09:11:42Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--PyMOL|Česká verze--> =Description= PyMOL is a Python-enhanced molecular graphics tool. It excels at 3D visualization of proteins, small molecules, density, surfaces, and trajectories. It also includes molecular editing, ray tracing, and movies. =Availability= Version 1.7.6. Freely available to all users. Modules: pymol-1.7.6-gcc =Licence= [https://docs.python.org/2/license.html CNRI Python Licence] =Use= Load and run example: module add pymol-1.7.6-gcc pymol {{Application with GUI}} =Documentation= Documentation is available at http://www.pymolwiki.org/index.php/Main_Page . =Program manager= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://sourceforge.net/projects/pymol/ [[Category:Applications]] [[Category:English]] ravgofxq410hrggydgsfqje8ymu4zhi 0 381 5182 4870 2016-05-13T09:15:05Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--RSeQC|Česká verze--> = Description = RSeQC evaluates different aspects of RNA-seq experiments, such as sequence quality, GC bias, polymerase chain reaction bias, nucleotide composition bias, sequencing depth, strand specificity, coverage uniformity and read distribution over the genome structure. RSeQC takes both SAM and BAM files as input, which can be produced by most RNA-seq mapping tools as well as BED files, which are widely used for gene models. Most modules in RSeQC take advantage of R scripts for visualization, and they are notably efficient in dealing with large BAM/SAM files containing hundreds of millions of alignments. = Availability = Versions: * 2.6.1 (module <code>RSeQC-2.6.1</code>) = Use = Load module: <source lang="bash"> $ module add RSeQC-2.6.1 </source> After it you can type: <code>python -c ‘from qcmodule import SAM’</code>. If no error message comes out, RSeQC modules have been installed successfully. = Documentation = * Online documentation [http://rseqc.sourceforge.net/ http://rseqc.sourceforge.net/]. * Article about application and used algorithms [http://www.ncbi.nlm.nih.gov/pubmed?term=%2022743226 RSeQC: quality control of RNA-seq experiments.] = Licence = STAR is a software distributed under GPLv3. = Program administrator = * [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = URL: [http://rseqc.sourceforge.net/ RSeQC: An RNA-seq Quality Control Package] [[Category:Applications]] [[Category:English]] bb9xsyxctof0pripocwa12ynbstz1we 0 387 4874 4838 2016-05-13T09:09:45Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--XMDS|Česká verze--> = Description = XMDS is a software package that allows the fast and easy solution of sets of ordinary, partial and stochastic differential equations, using a variety of efficient numerical algorithms. = Availability = Version 2.2.2. Modules available: xmds2-2.2.2 xmds2-2.2.2-serial = Licence = [http://www.gnu.org/licenses/gpl-2.0.html GPLv2] = Use = module add xmds2-2.2.2 module add xmds2-2.2.2-serial = Documentation = http://www.xmds.org/tutorial.html = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://www.xmds.org/index.html [[Category:Applications]] [[Category:English]] d34rc3bi25ri7c412ww3r0yx6w0g253 0 388 5085 5080 2016-05-13T09:13:39Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--PHP|Česká verze--> =Description= Scripting language PHP is mostly used for managing web pages with dynamic content. There are many libraries which extend base functionality of the language. =Availability= Available in module: * Module <code>php-5.6.11</code>: PHP version 5.6.11 (10.7.2015) =Use= Steps to run the application <source lang="bash"> $ module add php-5.6.11 $ php name_of_the_script.php </source> The PHP memory limit is implicitly set to 128 MB. In case of need you can increase the limit by parameter of the command php. Also you can use variable of PBS to get allocated memory by a job: <source lang="bash"> $ php -d memory_limit=$TORQUE_RESC_MEM name_of_the_script.php </source> Example of the script: <source lang="php"> <?php // no time restriction, implicitly 30 second set_time_limit(0) // increase memory limit to 1 GB, implicitly 128 MB ini_set('memory_limit','1024M'); $variable = 5 + 3; echo "Sum of 5 + 3 = $variable "; </source> =Documentation= [http://php.net/manual/en/ Documentation] =Licence= The PHP License. Available for all users. =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= URL: http://www.php.net [[Category:Applications]] [[Category:English]] 2qb6kt9af6bnrs90x97ciju7bzvkutk 0 389 5206 5130 2016-05-13T09:15:29Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--PIConGPU|Česká verze--> = Description = PIConGPU is a relativistic Particle-in-Cell (PIC) code running on graphic processing units (GPUs). = Availability = Version 0.1.0. Modules available: picongpu-0.1.0 = Licence = [http://www.gnu.org/licenses/gpl-3.0.html GPLv3+] = Use = module add picongpu-0.1.0 = Documentation = https://github.com/ComputationalRadiationPhysics/picongpu/blob/master/examples/KelvinHelmholtz/documentation.yml = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = https://www.hzdr.de/db/Cms?pNid=3227 [[Category:Applications]] [[Category:English]] 5fh39exkyegjslnav6jtcwz240xltno 0 391 5198 5185 2016-05-13T09:15:18Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--SAMtools|Česká verze--> =Description= VSEARCH is an alternative to [http://www.drive5.com/usearch/ USEARCH]. It should: have a 64-bit design that handles very large databases and much more than 4GB of memory be as accurate or more accurate than usearch be as fast or faster than usearch =Availability= Version 1.4.4 =Use= module add vsearch-1.4.4 for more details see Documentation. =Documentation= https://github.com/torognes/vsearch/blob/master/README.md =Program manager= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= https://github.com/torognes/vsearch/<br /> [[Category:Applications]] [[Category:English]] 7dyye3xqrn8lozh4jn8b1wr30wux68n 0 392 5188 5178 2016-05-13T09:15:09Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--ASTRID|Česká verze--> ===Description=== Accurate Species TRee Reconstruction with Internode Distances ===Availability=== Program is available in version 1.0 ===Use=== module add astrid-1.0 ASTRID ===Documentation=== http://pranjalv123.github.io/ASTRID/ ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://pranjalv123.github.io/ASTRID/ [[Kategorie:Applications]] mlnjnvjctlvo7szebk4s8udturn290u 0 393 4949 4886 2016-05-13T09:11:16Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--ExaML|Česká verze--> = Description = Exascale Maximum Likelihood (ExaML) code for phylogenetic inference using MPI. = Availability = Version 3.0.15 = Use = module add examl-3.0.15 examl -h for more info = Documentation = https://github.com/stamatak/ExaML/blob/master/manual/ExaML.pdf = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = https://github.com/stamatak/ExaML [[Category:Applications]] [[Category:English]] 20n9bwhumwteizwqgkn6d5ua2ui1ssl 0 396 5103 4982 2016-05-13T09:13:56Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--Guidance|Česká verze--> =Description= GUIDANCE is meant to be used for weighting, filtering or masking unreliably aligned positions in sequence alignments before subsequent analysis. Program can also use modules: mafft, clustalw, muscle, bioperl =Availability= Version 2.0 =Use= module add guidance-2.0-gcc / guidance-2.0-intel =Licence= Licence notes that the use of the program is for academic use only =Program manager= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://guidance.tau.ac.il/ver2/ [[Category:Applications]] [[Category:English]] bjdjffvie5o0um4dc8rm1bv1mfj16sm 0 398 907 906 2016-01-23T18:50:48Z Root 1 naimportována 1 revize wikitext text/x-wiki ===Description=== RINS (Rapid Identification of Non-human Sequence) is a software tool that detects non-human sequences (virus or other micobails) from human RNA-seq data This tool could also be used to detect non-X species's sequence from X RNA-seq data if corresponding dataset is provided ===Availability=== Program is available in version 1.0 ===Use=== Initialize enviroment ("module add rins") and run "rins.pl". Config.txt file is required and available in directory /software/rins/1.0/rins/ ===Documentation=== http://khavarilab.stanford.edu/tools-1/#tools ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://khavarilab.stanford.edu/tools-1/#tools [[Kategorie:Applications]] h0dzfyfprhd94purse9ul4t3zknqrsa 0 399 909 908 2016-01-23T18:50:48Z Root 1 naimportována 1 revize wikitext text/x-wiki =Description= FastStructure is an algorithm for inferring population structure from large SNP genotype data. It is based on a variational Bayesian framework for posterior inference and is written in Python2.x. Here, we summarize how to setup this software package, compile the C and Cython scripts and run the algorithm on a test simulated genotype dataset. =Availability= Appliacation is available for all MetaCentrum users. Versions: * 1.0 in module <code>faststructure-1.0</code> =Use= Steps to run the application module add faststructure-1.0 structure.py The application can be run by command <code>python structure.py</code> according to the official fastStructure manual. In MataCentrum you can run it only by <code>structure.py</code>, because we made the file runnable. If you would like to use original syntax, you should type whole path to the file (<code>python /software/faststructure/1.0/structure.py</code>). =Documentation= Documentation can be found at http://rajanil.github.io/fastStructure/ =Licence= The MIT License (MIT) =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= URL: http://rajanil.github.io/fastStructure/ [[Category:Applications]] [[Category:Aplikace]] [[Category:English]] a3pj2fnfqexmqukvbagh4c7v5edcrn3 0 400 5121 5055 2016-05-13T09:14:10Z Root 1 Nahrazení textu „([[“ textem „<!--“ wikitext text/x-wiki <!--SOLpro|Česká verze--> ===Description=== SOLpro predicts the propensity of a protein to be soluble upon overexpression in E. coli using a two-stage SVM architecture based on multiple representations of the primary sequence. Each classifier of the first layer takes as input a distinct set of features describing the sequence. A final SVM classifier summarizes the resulting predictions and predicts if the protein is soluble or not as well as the corresponding probability. ===Availability=== Only one unnumbered version from 2009. solpro ===Use=== module add solpro predict_sol.sh input output ===Documentation=== http://scratch.proteomics.ics.uci.edu/explanation.html#SOLpro http://download.igb.uci.edu/ ===Licence=== Download SOLpro (free for academic, non commercial, use) ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://scratch.proteomics.ics.uci.edu/explanation.html#SOLpro [[Kategorie:Applications]] dw3gjjqr1pi7dzslmgrwdhvu3e616tm 0 401 913 912 2016-01-23T18:50:49Z Root 1 naimportována 1 revize wikitext text/x-wiki ===Description=== RapMap is a testing ground for ideas in read mapping. ===Availability=== Program is available in version 0.2.0 ===Use=== Initialize enviroment ("module add rapmap-0.2.0") and run rapmap -h . ===Documentation=== https://github.com/COMBINE-lab/RapMap/blob/master/README.md ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== https://github.com/COMBINE-lab/RapMap [[Kategorie:Applications]] 7sg4y73uag86s5mjbw0mpt0n9zh42ai 0 402 915 914 2016-01-23T18:50:49Z Root 1 naimportována 1 revize wikitext text/x-wiki ===Description=== HISAT is a fast and sensitive spliced alignment program. ===Availability=== Program is available in version 0.1.7 ===Use=== Initialize enviroment ("module add hisat-0.1.7") and run hisat -h . ===Documentation=== https://github.com/infphilo/hisat/blob/master/MANUAL ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== https://github.com/infphilo/hisat [[Kategorie:Applications]] 3ft015pniy319qe0s1le747ovene1wm 0 537 5240 2044 2016-05-13T14:28:54Z Root 1 wikitext text/x-wiki ==Own compilations== If one needs to perform a '''non resource-consuming compilation''' (i.e., not lasting several hours), the '''[[Frontend|frontends]] could be used''' for such a compilation. If a more '''resource-consuming compilation''' is needed, the '''[[How to compute/Interactive jobs|interactive jobs]]''' asking for a computing node in an interactive mode '''should be used'''. '''''Note:''' All of our nodes provide the same SW equipment (with slight exceptions only) -- not including the nodes being upgraded to newer Debian version. Thus, the code compiled on a chosen node should be running on each infrastructure node.'' Such an application could be then used '''directly from your home directory''' or -- if more users could be interested in the application -- '''it could be moved to the common application location''' (the installation and module will then be prepared, as indicated in the following section). ==Preparation of a new module== {{Warning | ''Since March 2014, we started to use the so-called '''AFS read-only replicas''' for the software volumes in order to improve the reliability and accessibility of the software modules -- the basic directory <code>/software/NAME-1.1</code> becomes '''available read-only''', while the '''new working directory for installation/updating the SW becomes <code>/afs/.ics.muni.cz/software/NAME-1.1/</code>'''! (The working volume has to be '''released''' in order to make it available via read-only replicas -- see manual below...)''}} If you wish to install your own application, or one that is not in our list of applications, and later manage such application, then: # '''ask at''' [mailto:meta@cesnet.cz meta@cesnet.cz] '''for preparation of an AFS volume''', specify your login (or logins of your colleagues that will manage application with you), name, version and expected size of application # once the AFS volume becomes created, '''you''ll be able to to install the application''' into a folder, e.g <code>/afs/.ics.muni.cz/software/NAME-1.1</code> # the '''ideal structure of the installation folder''' is (it is mostly achieved by specification of <code>--prefix=/software/NAME-1.1</code> to the <code>configure</code> command): #*'''bin''' - binaries #*'''lib''' - libraries #*'''include''' - headers #*'''man''' - man pages #*'''doc''' - documentation #*'''examples''' - examplex #*'''src''' - source codes # for the compilation, the '''compilers available via modules''' (e.g., gcc-4.5.2, pgicdk-13.10, intelcdk-14, ...) and relevant MPI runtime/libraries (e.g., openmpi, openmpi-1.6-pgi,openmpi-1.6-intel, mpich3-xxx, ...) are comonly used # '''note the installation steps''' into some file, e.g. <code>/afs/.ics.muni.cz/software/NAME-1.1/howto_install</code> (it may become useful in case of upgrade) # once you successfully test the application, perform so-called '''volume release''' -- a transformation of the read-write AFS volume into read-only replicas, which increase the availability and reliability of the SW volumes in cases of specific failures. #* simultaneously, the main software directory become protected from accidential changes (in common case, one should start the applications from the read-only replicas, available through <code>/software/NAME-1.1/</code>) #* perform the command '''<code>remctl kdccesnet.ics.muni.cz afs release soft.NAME-1.1</code>''' #** write us in cases of a permission failures #* the replicas availability could be checked by the command <code>fs where /software/NÁZEV-1.1</code> # once the release process sucessfully finishes, '''prepare a modulefile for your application''', and send it back to us (or make available somewhere) #* let be inspired by modules available in <code>/afs/ics.muni.cz/packages/amd64_linux26/modules-2.0/modulefiles/</code> #* if you want to add some help to module, available then via 'module help' command, see similar files in <code>/afs/ics.muni.cz/packages/amd64_linux26/modules-2.0/helpfiles</code> directory #* if you don't know how to create the module, we'll prepare it for you (in such a case, inform us about the environtment variables and modules your application requires for proper run) # at the end, '''write documentation on wiki''' [[MetaCentrum_Application_List]] with [[Format_for_application_description]] {{Rámeček-green|text='''''Note:''' Feel free to '''[[:Category:User support|write us]] in case of problems''' &ndash; we'll try to help you...''}} == Perl libraries == First check if there is an already installed perl package in one of our [[Perl - modules|modules]]. === easy way - using CPAN tool === Using the CPAN tool is quite effective way to install perl packages. For better effectivity it needs some configuration first. Example: perl -MCPAN -e shell cpan>o conf makepl_arg PREFIX=/afs/.ics.muni.cz/software/bioperl/1.6.9/gcc o conf mbuildpl_arg "--prefix /afs/.ics.muni.cz/software/bioperl/1.6.9/gcc" o conf build_dir /scratch/$LOGIN/.cpan/build o conf commit * Prefix settings are according to the program directory you are trying to install the software in. * Build directory should be on some fast disk, so the best way is to use /scratch * To acquire information about all CPAN tool settings issue the '''<tt>o conf</tt>''' command without parameters. The module installation is then simple: '''m /regexp/''' - list available modules matching case insensitive ''regexp'' or accurate module info '''install module_name''' - installs the module with all dependencies '''force install module_name''' - continues even if error occurs '''?''' - help use the CPAN tool '''q''' - exit the CPAN tool '''Don't forget''' to properly set the $PERL5LIB environment variable pointing to ''lib'' directory of installed module. In our case it is: export PERL5LIB=/software/bioperl-1.6.1/lib/perl/5.10.1:$PERL5LIB === hard way - manual package installation === If everything goes wrong, you have the option to install the packages manually. This means to download a package and all of it's dependencies and than install it in the proper order. TOIN=/software/EXPECTED_FOLDER perl Makefile.PL PREFIX=$TOIN make make test make install to test export PERL5LIB=$TOIN/lib/perl/5.10.1 perl > use My::Module; ==Python packages == === easy way - using PyPI tool === There are some different PyPI installations. You can use the 'pip' command installed on every [[frontend]] and tied with system Python version. Or you can use the PyPI that we have prepared in following modules: python26-modules-gcc python26-modules-intel You can just add one of these modules and than install required package (see bellow). If you don't like our PyPI installation and want your own one, just get PiPY from [http://www.pip-installer.org/en/latest/installing.html here] and then invoke python get-pip.py --root /some/new/directory/with/modules ==== PyPI operation ==== pip search <module name> pip install <module name> --root /some/python/modules/folder pip install git+https://path/to/git/file Ideally the <tt>root</tt> is in case of use of one of ours python26-modules the path <tt>/software/python26-modules</tt> '''Don't forget''' to properly set the $PATH and $PYTHONPATH environment variables if you are not using one of ours python26-modules and installing modules to some new dir. For details see the [[#hard way - manual package installation| hard way]] chapter. Sometimes help to specify more options, eg. module add python27-modules-gcc pip install -v --upgrade dendropy --install-option="--prefix=/afs/.ics.muni.cz/software/python27-modules" --root /afs/.ics.muni.cz/software/python27-modules --ignore-installed === Detailed walkthrough - using PyPI tool === A very convenient feature is to use the --user option of pip install. This will install modules, additional to the available system python install, in the location defined by the PYTHONUSERBASE environment variable. A convenient choice for this variable is a location visible from the NFSv4 infrastructure, which means you could use for example export PYTHONUSERBASE=/storage/home/<user_name>/.local If you install such modules at this location, you will also need to add them to your path and pythonpath so that they are accessible from any folder you wish to execute your code. For this purpose, export PATH=$PYTHONUSERBASE/bin:$PATH and export PYTHONPATH=$PYTHONUSERBASE/bin:$PYTHONPATH will do the job. If you wish to execute such commands at each login on a front end, you will therefore have to add the following lines to you .profile: module add python27-modules-intel # Set pip path for --user option export PYTHONUSERBASE=/storage/plzen1/home/<user_name>/.local # set PATH and PYTHONPATH variables export PATH=$PYTHONUSERBASE/bin:$PATH export PYTHONPATH=$PYTHONUSERBASE/lib/python2.7/site-packages:$PYTHONPATH With this, you can install any module you need with the following command: pip install <module-name> --user --process-dependency-links without any need for administrator rights, and you will be able to use it. When launching jobs from the scheduler, remember that you .profile is not executed, you will therefore need to do module add and to define the relevant environment variables before the job is acutally executed. === hard way - manual package installation === If everything goes wrong, you have the option to install the packages manually. This means to download a package and all of it's dependencies and than install it in the proper order. TOIN=/software/EXPECTED_FOLDER python setup.py install --install-scripts=$TOIN/bin/ --install-purelib=$TOIN/lib --install-lib=$TOIN/lib to test instalation export PYTHONPATH=$TOIN/lib:$PYTHONPATH export PATH=$TOIN/bin:$PATH python -c "import package;" sometimes is necessary to export PYTHONPATH before installation, but it must be definitely exported before using of package. ==R packages== The procedure for R packages installation is described at the [[R application#FAQ (usefull links)|R module page]]. [[Category:Applications]] tdhagmb84cf9yviqgb6j6kj8ydtebsn 0 564 5271 5267 2016-05-14T14:26:46Z Root 1 Nahrazení textu „Scheduled jobs“ textem „Batch jobs“ wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[How to compute]]|}} {{BoxPoložka|[[Working with data]]|}} }} ([[Spuštění aplikace na platformě JAVA|Česká verze)]] This page shows how to run an application on JAVA platform. Use the application [[JDK application|JDK]] for running an application on JAVA platform. ==Java parallelization== ===No parallelization=== If your application doesn't use ''threads'' everything is done in ''main'' thread without any parallelization. You can't usually control threads in an application, which is not yours. You can [[How to compute/Requesting resources|request]] the whole machine via the '''#excl''' option to solve this problem. ===Thread parallelization=== {{YellowInfo|If you would like to learn more about threads, see [http://docs.oracle.com/javase/tutorial/essential/concurrency/ The Java Tutorials - Lesson: Concurrency].}} You can easily use multiple CPUs on a same machine with JAVA platform because JAVA supports [https://en.wikipedia.org/wiki/Thread_%28computing%29 threads]. It is important to realize that one machine is used by multiple users who are supposed not to exceed given resources. JAVA allows to control number of running threads fairly well, make sure you won't run more threads than number of CPUs you requested. If you requested a whole machine via '''#excl''', you can get number of CPUs in a machine by calling method {{JavaMethod|package=java/lang|class=Runtime|method=availableProcessors()}}. Otherwise you have to work with the given number of CPUs if you requested a limited amount of CPUs. You can call method {{JavaMethod|package=java/util/concurrent|class=Executors|method=newFixedThreadPool(int)}} to force using '''n''' threads at maximum. See [http://docs.oracle.com/javase/tutorial/essential/concurrency/pools.html Thread Pools] for more information. The following example shows a program using specified number of threads: <source lang="java"> import java.util.Arrays; import java.util.List; import java.util.concurrent.*; public class Exekuce { static public class Producer implements Callable<String> { private final int n; public Producer(int n) { this.n = n; } public String call() throws Exception { //a difficult computation return " Product("+n+")"; } } public static void main(String[] args) throws ExecutionException, InterruptedException { //number of threads by parameter if possible, if not by number of CPUs int numThreads = Runtime.getRuntime().availableProcessors(); if(args.length>0) { numThreads = Integer.parseInt(args[0]); } //thread pool of specified size ExecutorService executorService = Executors.newFixedThreadPool(numThreads); List<Producer> producers = Arrays.asList(new Producer(1), new Producer(2), new Producer(3)); //parallel processing List<Future<String>> futures = executorService.invokeAll(producers); executorService.shutdown(); //get results for (Future<String> fs : futures) { System.out.println("výsledek " + fs.get()); } } </source> ==Running a JAVA program== '''1.''' : [[How to compute/Accessing machines|Login]] to any available [[Frontend|frontend]], except of '''skirit'''. '''2.''' : Prepare your program Prepare following file in you home directory (e.g. ''/storage/plze1/home/$LOGNAME'' '''HelloWorld.java''' <source lang="java"> public class HelloWorld { public static void main(String args[]) { System.out.println("Hello world " + args[0]); } } </source> load [[JDK application|JDK]] modul module add jdk-8 and compile it with '''javac''' command: javac HelloWorld.java -> ''HelloWorld.class'' file is created. '''3.''' : Prepare your startup (batch) script {{Box|Related topics {{BoxPoložka|[[How to compute/Batch jobs|Batch jobs]]|}} }} [[JDK application|JDK]] offers whole Java Development Kit (JDK), including Java Runtime Environment (JRE) through the '''java''' command. Prepare following file as your startup script. '''task.sh''' <source lang="bash"> #!/bin/bash module add jdk-8 # loads JDK module java -cp /storage/plzen1/home/$LOGNAME HelloWorld Hello </source> '''4. ''': Submit your job Submit the job with following command (see [[How to compute/Batch jobs|Batch jobs]] for more info)): qsub -q short -l nodes=1:ppn=1:nfs4,mem=1gb -j oe -m e task.sh You will be noticed when the job is finished via email and you will find results of your program in the directory you submitted the job from (''/storage/plzen1/home/$LOGNAME'' in this example). Results are saved in a file with name similar to ''task.sh.o12345''. [[Category:Applications]] [[Category:Grid services]] c9n8blrbgmcmy5ouc81obdgos1n36nn 0 318 4783 4761 2016-05-13T09:07:46Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--DAMASK|English version--> = Description = At the core of DAMASK is a flexible and hierarchically structured model of material point behavior for the solution of (thermo-) elastoplastic boundary value problems. Its main purpose is the simulation of crystal plasticity within a finite-strain continuum mechanical framework. = Availability = Version 2689. Freely available to users. Modules: damask-2689 = Licence = [http://www.gnu.org/copyleft/gpl.html GPLv3] = Use = Initialize environment with command: module add damask-2689 {{MPI application}} = Documentation = Documentation is at http://damask.mpie.de/Documentation/WebHome. = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://damask.mpie.de/ [[Category:Applications]] [[Category:English]] 20dhiya53s6bcbsphqvdpc1gxgr3wcq 0 356 4800 4728 2016-05-13T09:08:04Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Trinotate|Česká verze--> = Description = Trinotate is a comprehensive annotation suite designed for automatic functional annotation of transcriptomes, particularly de novo assembled transcriptomes, from model or non-model organisms. Trinotate makes use of a number of different well referenced methods for functional annotation including homology search to known sequence data (NCBI-BLAST), protein domain identification (HMMER/PFAM), protein signal prediction (singalP/tmHMM), and comparison to currently curated annotation databases (EMBL Uniprot eggNOG/GO Pathways databases). All functional annotation data derived from the analysis of transcripts is integrated into a SQLite database which allows fast efficient searching for terms with specific qualities related to a desired scientific hypothesis or a means to create a whole annotation report for a transcriptome. = Availability = Version from 20140708 = Licence = GPL = Use = module add trinotate-201407 trinotate It includes modules jdk-8, hmmer-3.1b1-gcc, trinity-201404, blast+-2.2.29 and perl-5.10.1 = Documentation = http://trinotate.sourceforge.net/ = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://trinotate.sourceforge.net/ [[Category:Applications]] [[Category:English]] [[Kategorie:Aplikace]] [[Kategorie:Česky]] 0exm2qoon65og44vfn9y45mjgczvm6u 0 246 4806 4741 2016-05-13T09:08:17Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Elk_FP-LAPW|Česká verze--> ===Description=== '''Elk FP-LAPW''' is an All-Electron Full-Potential Linearised Augmented-Plane Wave code for determining the properties of crystalline solids. ===Availability=== * '''Module <code>elk-2.2.9</code>:''' ''Elk FP-LAPW version 2.2.9'', 64-bit version, supporting OpenMP and MPI, compiled against the Intel MKL and FFTW3 libraries. * ''Module <code>elk-1.4.22</code>:'' ''Elk FP-LAPW version 1.4.22'', 64-bit version, supporting OpenMP and MPI, compiled against the Intel MKL library. ===Use=== '''1. Running the application -- ''Interactive mode:''''' * ask the scheduler for an interactive job having a desired number of nodes (<code>nodes</code> attribute) and a desired number of processors (<code>ppn</code> attribute) reserved <source lang="bash"> $ qsub -I -l nodes=X:ppn=Y -l mem=Zg </source> :''Note: Do not forget to apppropriately set the amount of requested memory (<code>mem</code> attribute) and/or another job requirements.'' * load the application module <source lang="bash"> $ module add elk-2.2.9 </source> * change the working directory to the one containing the ELK input file and run the computation ** '''parallel computation on a single node (OpenMP)''': <source lang="bash"> $ cd $SCRATCHDIR/my_computation $ elk >computation.log </source> ::* '''''Note:''' Check the setting of the <code>OMP_NUM_THREADS</code> environment variable -- for such a computation, the variable should be set to the number of dedicated processors on a node (<code>Y</code> -- see above).'' :* '''distributed computation combined with parallel run (MPI + OpenMP)''': <source lang="bash"> $ cd $SCRATCHDIR/my_computation $ mpirun -pernode elk >computation.log </source> ::* '''''Note:''' Check the setting of the <code>OMP_NUM_THREADS</code> environment variable -- for such a computation, the variable should be set to the number of dedicated processors on a node (<code>Y</code> -- see above).'' :* '''distributed computation through multiple nodes (MPI only processes)''': <source lang="bash"> $ cd $SCRATCHDIR/my_computation $ export OMP_NUM_THREADS=1 $ mpirun elk >computation.log </source> ::* '''''Note:''' Check the setting of the <code>OMP_NUM_THREADS</code> environment variable -- for such a computation, the variable should be set to 1.'' '''2. Running the application -- ''Batch mode:''''' * prepare the job description script -- use a general skeleton supplemented by the following lines: <source lang="bash"> ... # load the application module module add elk-2.2.9 # change the working directory to the one containing the ELK input file and run the computation cd $SCRATCHDIR/my_computation elk >computation.log # parallel computation on a single node mpirun elk >computation.log # distributed computation through multiple nodes (see other variants above) ... </source> * pass the job description file to the scheduler together with (at least) the requested number of nodes, processors and requested amount of memory <source lang="bash"> $ qsub -l nodes=X:ppn=Y -l mem=Zg mydescriptionscript.sh </source> ===Documentation=== The documentation is available either at [http://elk.sourceforge.net/#documentation producer's webpage] ([http://elk.sourceforge.net/elk.pdf direct link to PDF manual]) or locally in the program directory (<code>/software/elk-1.4.22/docs/</code>). ===License=== open-source under terms of the [http://www.gnu.org/licenses/gpl.txt GNU General Public License version 3] ===Supported platforms=== amd64 ===Program administrator=== Tom Rebok [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://elk.sourceforge.net [[Kategorie:Applications]] 4vmrfsu4n01mx0u7vw6bwk4nxrkjtis 0 233 4814 4779 2016-05-13T09:08:26Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--ClustalOmega|Česká verze--> ===Description=== Clustal-Omega is a general purpose multiple sequence alignment (MSA) program for proteins and DNA/RNA. It produces high quality MSAs and is capable of handling data-sets of hundreds of thousands of sequences in reasonable time. ===Availability=== Version 1.1.0 ===Use=== module add clustal-omega-1.1.0 clustalo see Documentation. ===Documentation=== http://www.clustal.org/omega/ ===Licence=== GPL version 2 ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.clustal.org/omega/ [[Kategorie:Applications]] iafb5i765vt17x2w6w99ay7lhepfken 0 150 4817 4769 2016-05-13T09:08:33Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Gaussian-GaussView|Česká verze--> ===Description=== Gaussian is package of programs based on basics of quantum mechanics. Program serves to predict the energies, molecular structures, and vibrational frequencies of molecular systems, along with numerous molecular properties derived from these basic computation types. It can be used to study molecules and reactions under a wide range of conditions, including both stable species and compounds which are difficult or impossible to observe experimentally such as short-lived intermediates and transition structures. ===Availability=== Program is actually available in many versions, older Gaussian 98 (g98), newer Gaussian 2003 (g03-D.01 and g03-B.05), revision E.01 (g03), g09-A.02 including GaussView (only jcu users), g09-A02 and the newest Gaussian 09 ('''g09''', '''G09-C.01''' and '''G09-D.01'''). Module '''g09-D.01linda''' contains Linda version allowing to use more machines. '''If you do not use Linda version, other versions permit computing only on one machine''', thus it is nonsense to plan computation on more machines. Although job submission in form nodes=2:ppn=x will not end with an error, it will compute only on one machine like you submit it with <tt>nodes=1</tt> parameter. This results in overcomming the requested resources on memory or scratch size and the job would be killed. {{Warning | The purchased licenses permit '''just an academic use''' of the program!}} ===Use=== '''''Hint:''' for data-intensive computations, there is a big scratch space available from the mandos cluster (<code>/scratch.shared</code>) -- the volume is shared for all the mandos nodes.'' * '''prepare the Gaussian input file''', where you should specify the intended computation ** copy the file, for example, to your home directory on a shared NFSv4 volume (volumes in the directory <code>/storage</code>) * after that, choose whether you want to use Gaussian in a ''batch'' or ''interactive mode'': '''''1. Batch computation:''''' * '''prepare the job description script''', where you should specify the computation, which should be performed over the input file ** '''''Recommendation:''' we highly recommend to use the <code>g09-prepare</code> utility from within the script -- within the file, the utility automatically sets the parameters <code>%nproc</code> (number of processors), <code>%mem</code> (amount of usable memory) and <code>%rwf</code> (amount of scratch space -- for large computations, the scratch space may also be separated into multiple files, which may be automatically erased by Gaussian during the computation) based on the real resources dedicated for the job. See <code>g09-prepare -h</code>''. ** for example, such a script may look like this one: <source lang="bash"> #!/bin/bash #ensure removing the temporary data if the job ends or fails trap "clean_scratch" TERM EXIT DATADIR="/storage/brno2/home/$LOGNAME/" # copy input file from shared network disk to local disk cp $DATADIR/myinput.com $SCRATCHDIR || exit 1 cd $SCRATCHDIR || exit 2 # let's load the Gaussian module module add g09 # let the resource requirements (specified within the input file) be automatically set so that the correspond to the resources reserved g09-prepare myinput.com # start the computation g09 myinput.com # alternatively: g09 <myinput.com >myoutput.out # copy the output from local scratch to shared network disk cp myoutput.out $DATADIR || export CLEAN_SCRATCH=false </source> * '''pass the startup script to the scheduler''' together with resource requirements (number of processors, amount of usable memory, scratch size, etc.) ** ''Note: The Gaussian version purchased by MetaCentrum does not support a distributed run over a multiple nodes; thus, use just a single node with multiple processors for your computations...'' <source lang="bash"> $ qsub -l nodes=1:ppn=X -l mem=Yg -l scratch=Z mydescriptionscript.sh </source> '''''2. Linda computation:''''' (using more machines) The script remains the same, we only change ''module add g09'' for ''module add g09-D.01linda'' and submit the job with given number of machines V <source lang="bash"> $ qsub -l nodes=V:ppn=X -l mem=Yg -l scratch=Z mydescriptionscript.sh </source> '''''3. Interactive computation:''''' * ask for an interactive job having a single node with a specified number of processors (<code>ppn</code> parameter), specified amount of memory (<code>mem</code> parameter), and specified amount of scratch space (<code>scratch</code> parameter -- in KB) reserved: <source lang="bash"> $ qsub -I -l nodes=1:ppn=X -l mem=Yg -l scratch=Z </source> :''Note: If necessary, specify also another job requirements...'' * load the Gaussian module into the environment <source lang="bash"> $ module add g09 </source> * prepare the Gaussian input file (named e.g. <code>myinput.com</code>) * '''''Recommendation:''' we highly recommend to use the <code>g09-prepare</code> utility to automatically set the input file's parameters <code>%nproc</code> (number of processors), <code>%mem</code> (amount of usable memory) and <code>%rwf</code> (amount of scratch space -- for large computations, the scratch space may also be separated into multiple files, which may be automatically erased by Gaussian during the computation) based on the real resources dedicated for the job. See <code>g09-prepare -h</code>''. <source lang="bash"> $ g09-prepare myinput.com </source> * run the computation <source lang="bash"> $ g09 myinput.com # alternatively: $ g09 <myinput.com >myoutput.out </source> ===Documentation=== [http://www.gaussian.com/g_tech/g_ur/g09help.htm On-line] documentation is available. ===Licence=== Gaussian 03 and 09 can be used on the clusters installed in Brno (Supercomputing Centre MU) location. Everybody interested in using this program at another location has to send his licence to the email: [mailto:meta@cesnet.cz meta@cesnet.cz], then we check authenticity of licence and enable program to the user. ===Supported platforms=== Linux ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] Domovská stránka URL: http://www.gaussian.com/ ===FAQs=== '''''Q:''' How to tell Gaussian to limit the size of the used /scratch? How to determine, what size had been really used?'' :'''A:''' Once a Gaussian computation finishes, it informs the user about the size of used files in the following form: File lengths (MBytes): RWF= 207 Int= 0 D2E= 0 Chk= 134 Scr= 1 It is possible to limit the amount of the used scratch space using the directives %RWF=,%Int=,%D2E= in the form (if more space is necessary, the computation crashes): %RWF=/directory/,200GB It is also necessary to tell PBS to reserve such a space on the [[Scratch mountpoint|scratch]] -- do it using the <code>-l scratch=XXX</code> parameter (see more information in [[Scratch mountpoint-->. The following example ensures an allocation of 200 GB space on the /scratch volume of the worker node:: -l scratch=200gb '''''To set the RWF parameter automatically (based on the resources really reserved for a job), please, use the <code>g09-prepare</code> command (see [[#Use|above-->.''' When using this script '''for large computations''', one may let the script to let the '''scratch space separate into multiple files''', which may be automatically erased by Gaussian during the computation (<code>%NoSave</code>).'' ''Note:'' Alternatively, the amount of scratch space may be limited using the MaxDisk=200GB parameter (however, this is just a hint for some computation methods, which are able to work with a disk using different ways -- generally, this parameter does not ensure exceeding of the specified value.) '''''Q''': Memory alocation in program Gaussian version G03.E01 does not work. What should I do?'' :'''A:''' If you get a warning: "buffer allocation failed in ntrext1." just after running the program, use as work directory $SCRATCHDIR instead of $HOME (there is an error in the new version, that occurs according type of file system) '''''Q''':I think there is a problem with the node orca14-2. Gaussian computings have fallen down immediately after running with error message (every time in different part of the log file) "Erroneous write. Write 8192 instead of 12288."'' :'''A:''' Reason of this failure is that your disk is full of undeleted Gaussian helping files. These files are remains from your previous computings which weren't successful. You can avoid this situation by deleting all helping files after the end of computing. See [[Working with data in job]]. '''''Q''': Computation runs only on one machine while multiple were requested.'' :'''A:''' Read again section [[#Availability|Availability]] '''''Q:''' Which settings are necessary for Linda version? :'''A:''' You should specify %NProcShared=2 %LindaWorkers=doom29,gram7 where the first defines how many cores you need per machine, and the second parameter is the list of machines. See http://www.gaussian.com/g_tech/g_ur/m_linda.htm . ''g09-prepare'' sets both parameters by default. [[Category:Applications]] 17483z102ymcpyxx8vty0lha3pxxhi9 0 336 4823 4766 2016-05-13T09:08:41Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Galaxy|Česká verze--> = Description = [[Soubor:Galaxylogo.png|x40px|link=http://galaxyproject.org/]] is an open, web-based platform for accessible, reproducible, and transparent computational biomedical research. * '''Accessible''': Users without programming experience can easily specify parameters and run tools and workflows. * '''Reproducible''': Galaxy captures information so that any user can repeat and understand a complete computational analysis. * '''Transparent''': Users share and publish analyses via the web and create Pages, interactive, web-based documents that describe a complete analysis. Our MetaCentrum Galaxy instance allows the analyses that are submited as regular jobs under real user's accounts (you can see jobs in [http://metavo.metacentrum.cz/pbsmon2/users/ Users page]). [https://wiki.metacentrum.cz/wiki/Kerberos_authentication_system Kerberos] is used for authentification by MetaCentrum accounts through Apache. [http://apps.man.poznan.pl/trac/pbs-drmaa/wiki DRMAA] connects Galaxy jobs with Torque. For downloading large files, Apache module Xsendfile is applied. [https://wiki.metacentrum.cz/metawiki/Galaxy Internal page of configuration] Current version is ''under construction'', if you find any mistakes send us error log and way how you get it. Also, if you want to integrate some tool (ideally one from [https://toolshed.g2.bx.psu.edu/ Galaxy tool shed]) in our Galaxy, let us know. = Availability = Version updated April 17 2014. = Licence = Licensed under the Academic Free License version 3.0 = Use = '''ALWAYS''' login through webpage https://galaxy.metacentrum.cz (old addresses https://galaxy.metacentrum.cz/login and https://galaxy.meta.zcu.cz/login will work too, but for the latter one, there isn't a valid ssl certifikacate for the page). It will create Kerberos tickets (that allows you to submit new analysis/job) and will redirect to .../galaxy URL. Later, if you plan to do next analyses after some time, use .../login URL again, because your tickets must be recreated. == Large or local files upload == Galaxy often applies FTP upload for large files. We apply standard user's folders in the storages (/storage/...) for such things, where you can easily upload your files (see [https://wiki.metacentrum.cz/wiki/Storage_types_-_tutorial Storage types tutorial] and [https://wiki.metacentrum.cz/wiki/File_systems_in_MetaCentrum File systems information]). In shortcut, if you need to upload large or local files into Galaxy: * (''do just once'') use eg. [https://wiki.metacentrum.cz/wiki/How_to_access_MetaCentrum_resources_from_Windows WinSCP] and connect to galaxy.metacentrum.cz (ensure that the SFTP file protocol is choosed - default) by your MetaCentrum login/password, create folder (e.g. galaxy-files) here and add read and execute permission for others and group (755) * (''do just once'') send email to [mailto:meta@cesnet.cz meta@cesnet.cz] with the folder name * eg. by [https://wiki.metacentrum.cz/wiki/How_to_access_MetaCentrum_resources_from_Windows WinSCP] you can easily copy your files in the folder and later you will see them under Galaxy in the "Upload file"-"Files uploaded via FTP" == Deleting files == We ask you for deleting old files in your histories. First delete file by click on cross close to file name then you can delete * files one by one by click on History options (sprocket icon) and then check Include deleted datasets, now, in the history, you see deleted datasets and you can delete them permanently, * all deleted files permanently by click on History options (sprocket icon) and then Purge deleted datasets. == Tutorials == Galaxy is widespread tool, therefore one can choose from many of tutorials to learn working with Galaxy, i.e. [https://wiki.metacentrum.cz/w/images/2/2b/METAgalaxy.pdf Our MetaCentrum tutorial],the main starting page is [https://wiki.galaxyproject.org/Learn Galaxy wiki] and [http://vimeo.com/galaxyproject Video tutorial page]. The starting guides * [http://vimeo.com/galaxyproject/upload Uploading data] * [http://vimeo.com/galaxyproject/fastqprep Fastq Illumina], [http://screencast.g2.bx.psu.edu/quickie_13_fastq_basic/flow.html Basic FASTQ manipulations], [http://screencast.g2.bx.psu.edu/quickie_14_fastq_adv/flow.html Advanced FASTQ manipulations] * [http://vimeo.com/galaxyproject/datasets1 Managing and dowloading data] * [http://nihlibrary.ors.nih.gov/bioinfo/ Detail step by step tutorials of many analyses with input data] * [http://training.bioinformatics.ucdavis.edu/docs/2013/05/galaxy-intro/workflows/ Screenshots for workflow learning] ===Step by step=== Easy start for [http://nihlibrary.ors.nih.gov/bioinfo/ngs/problem1.html Next Generation Sequencing Data Analysis]: * log in https://galaxy.metacentrum.cz/ by your MetaCentrum account * click Get data - Upload data * upload data from http://nihlibrary.ors.nih.gov/bioinfo/ngs/sourcedata1.zip (zipped file with 3 fastq reads and 1 fa - reference genome), download it, extract a upload to Galaxy * fastq grooming: click on menu NGS: QC and manipulation, choose FASTQ Groomer, leave defaults (choose 1 of fastq files from history, Sanger type), click execute * quality control: in the same section choose FastQC:Read QC, use groomed output, leave defaults, click execute * check quality by click on ''eye'' in the output of quality result in history * read mapping: click on menu NGS: Mapping, choose Map with BWA for Illumina, reference genome is one from history (fa file), for fastq file groomed fastq file, leave defaults (single-end) and click to execute * check the ouput (sam file) by click on ''eye'' on the last item in history * sam to bam conversion: in the menu NGS: SAM Tools, choose SAM-to-BAM, reference genome is one from history (fa file) and sam file is one after the mapping, click execute * download bam result from the history = Supported application = Actually we support * HelloMetaCentrum, it is a test job, that only stores information about machine where it was assigned. * [https://wiki.metacentrum.cz/wiki/Bowtie_application Bowtie2] available as module bowtie2-2.1.0, in menu NGS: Mapping - Bowtie2 * [https://wiki.metacentrum.cz/wiki/Bfast_application Bfast] available as module bfast-0.7.0, in menu NGS: Mapping - Map with BFAST * [https://wiki.metacentrum.cz/wiki/Blast_application Blast] available as module blast+-2.2.27, in menu NCBI Blast+ * [https://wiki.metacentrum.cz/wiki/BWA_application BWA] available as module bwa-0.7.5a, in menu NGS: Mapping - Map with BWA for Illuminaa * [https://wiki.metacentrum.cz/wiki/Cufflinks_application Cuff tools] available as module cufflinks-2.0.2, in menu NGS: RNA analysis - Cuff... * [https://wiki.metacentrum.cz/wiki/Python_-_moduly Cutadapt] available in module python27-modules-gcc, in menu NGS: QC and manipulation - Cutadapt * Fastq, Fastx, FastQC tools, in menu NGS: QC and manipulation - grooming, quality check, filtering of Fast... * [https://wiki.metacentrum.cz/wiki/Mosaik_application Mosaik] available as module mosaik-1.1, in menu NGS: Mapping - Map with Mosaik * [https://wiki.metacentrum.cz/wiki/Muscle_application Muscle] available as module muscle-3.8.31, in menu FASTA Manipulation - MUSCLE * [https://wiki.metacentrum.cz/wiki/RepeatExplorer_application RepeatExplorer] available as module repeatexplorerDEV, there are 2 days and 1 weak queues for clustering * [https://wiki.metacentrum.cz/wiki/RSEM_application RSEM] available as module rsem-1.2.8, in menu NGS: Mapping - RSEM * [https://wiki.metacentrum.cz/wiki/SAMtools_application SAMtools] available as module samtools-0.1.19, in menu NGS: SAM tools - Filter SAM or BAM, SAM-to-BAM and BAM-toSAM converts * [https://wiki.metacentrum.cz/wiki/TopHat_application TopHat2] available as module tophat-2.0.8, in menu NGS: RNA analysis Mapping - Tophat2 If not written something else, all jobs are send to the queue with 2 CPUs, using 10GB memory in 1 day queue. If you need other applications to include, write request to [mailto:meta@cesnet.cz meta@cesnet.cz] and look if it is in the [https://toolshed.g2.bx.psu.edu/ Galaxy tool shed]. = Documentation = https://wiki.galaxyproject.org/ = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = https://wiki.galaxyproject.org/ = Problems = Report the problems in detail, i.e. write which tool and input data you have used, what do you see behind icon of bug. * You do not see your Large files in FTP upload. Change permissions (og+rx)of your folder (and folder above) with files that you want to upload into galaxy. * galaxy_179.o: Permission denied If you get output of the job, where is written Permission denied many times, eg. Unable to copy file /var/spool/torque/spool/5764811.arien.ics.muni.cz.OU to /auto/plzen1/home/galaxy/galaxy-dist/database/job_working_directory/000/179/galaxy_179.o ** error from copy /bin/cp: accessing `/auto/plzen1/home/galaxy/galaxy-dist/database/job_working_directory/000/179/galaxy_179.o': Permission denied you should check that you have log in by ../login address before running of analysis. * IOError: [Errno 13] Permission denied If you see somehting like IOError: [Errno 13] Permission denied: '/storage/brno2/home/galaxy/tmp/GALAXY_VERSION_STRING_66' write email [mailto:meta@cesnet.cz meta@cesnet.cz], the error is hidden in the log files. [[Category:Applications]] [[Kategorie:Aplikace]] [[Category:English]] 98f3ebuwhnptrjz9aat04b1v5npca02 0 342 4824 4733 2016-05-13T09:08:42Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Jellyfish|Česká verze--> = Description = Jellyfish is a tool for fast, memory-efficient counting of k-mers in DNA. A k-mer is a substring of length k, and counting the occurrences of all such substrings is a central step in many analyses of DNA sequence. JELLYFISH can count k-mers quickly by using an efficient encoding of a hash table and by exploiting the "compare-and-swap" CPU instruction to increase parallelism. Jellyfish is a command-line program that reads FASTA and multi-FASTA files containing DNA sequences. It outputs its k-mer counts in an binary format, which can be translated into a human-readable text format using the "jellyfish dump" command. = Availability = Version 2.1.3. Freely available for every user. Version 1.1.11. This version works with [[Quake application]]. = Licence = [http://www.gnu.org/licenses/gpl-3.0-standalone.html GPL 3] = Use = Initialize module with command: module add jellyfish-2.1.3 module add jellyfish-1.1.11 = Documentation = Documentation is available in [ftp://ftp.genome.umd.edu/pub/jellyfish/JellyfishUserGuide.pdf JellyfishUserGuide.pdf] or it's local copy at <tt>/software/jellyfish-2.1.3/share/doc/JellyfishUserGuide.pdf</tt>. = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://www.genome.umd.edu/jellyfish.html [[Category:Applications]] [[Category:English]] mpsed4n5oygxn9bu50edj4lvli345im 0 259 4831 4729 2016-05-13T09:08:48Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--NCO|Česká verze--> ===Description=== NCO manipulates data stored in netCDF format. It also exploits the geophysical expressivity of many CF (Climate & Forecast) metadata conventions, the flexible description of physical dimensions translated by UDUnits, the network transparency of OPeNDAP, the storage features (e.g., compression, chunking, groups) of HDF (the Hierarchical Data Format), and many powerful mathematical and statistical algorithms of GSL (the GNU Scientific Library). ===Availability=== There is version 4.2.5 on MetaCentrum machines compiled by gcc with MPICH-3.0.2 support. ===Licence=== Free to whatever use. See [http://www.gnu.org/licenses/gpl-3.0.html GPLv3]. ===Use=== Initialize module with command module add nco-4.2.5-gcc After initialization are correctly set up the following system variables: $PATH, $NETCDF4_ROOT ===Documentation=== http://nco.sourceforge.net/#RTFM ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://nco.sourceforge.net/ [[Category:Applications]] 4oqdow7o3b2ubgqe71ear0aa8b8o444 0 236 4835 4737 2016-05-13T09:08:52Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Allpaths|Česká verze--> =Description= ALLPATHS is a whole genome shotgun assembler, that can generate high quality assemblies from very short reads. Assemblies are presented in a graph form that retains ambiguities, such as those arising from polymorphism, thereby providing information that has been absent from previous genome assemblies. =Availability= Version 2.2 and LG-48203. Available modules: allpaths-2.2 allpathslg-48203 =Use= module add allpaths-2.2 RunAllPaths or other binaries (see Documentation). =Documentation= * ftp://ftp.broadinstitute.org/pub/crd/ALLPATHS/Release-2-0/ * ftp://ftp.broadinstitute.org/pub/crd/ALLPATHS/Release-LG/AllPaths-LG_Manual.pdf (also available in directory <tt>/software/allpathslg-48203/doc</tt>) =Licence= * ftp://ftp.broadinstitute.org/pub/crd/ALLPATHS/Release-2-0/LICENSE * ftp://ftp.broadinstitute.org/pub/crd/ALLPATHS/Release-LG/LICENSE =Program manager= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= * ftp://ftp.broadinstitute.org/pub/crd/ALLPATHS/Release-2-0/ * http://www.broadinstitute.org/software/allpaths-lg/blog/ [[Category:Applications]] [[Category:English]] tdfsnjfvqkzwlsen6nqd290042pnbmg 0 261 4836 4773 2016-05-13T09:08:53Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Cuda_(NVidia)|Česká verze--> = Description = The NVIDIA® CUDA® Toolkit provides a comprehensive development environment for C and C++ developers building GPU-accelerated applications. The CUDA Toolkit includes a compiler for NVIDIA GPUs, math libraries, and tools for debugging and optimizing the performance of your applications. You’ll also find programming guides, user manuals, API reference, and other documentation to help you get started quickly accelerating your application with GPUs. = Availability = Freely available to all users. Modules: cuda -> cuda-7.5 cuda-3.2-kky cuda-4.0 cuda-4.1 cuda-4.2 cuda-5.0 cuda-5.5 cuda-6.0 cuda-6.5 cuda-7.0 cuda-7.5 = Licence = [http://docs.nvidia.com/cuda/eula/index.html#nvidia-cuda-toolkit-license-agreement Own] = Use = Add the module and use it: module add cuda = Documentation = http://docs.nvidia.com/cuda/#axzz3uZeYiz1h = Program administrator = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = https://developer.nvidia.com/cuda-toolkit [[Category:Applications]] [[Category:English]] qgdwoqu7nyire334zow1emw0b4fbzy3 0 376 4849 4738 2016-05-13T09:09:11Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Hashrf|Česká verze--> ===Description=== Software tool for computing Robinson-Foulds topological distance between phylogenetic trees. ===Availability=== hashrf-6.0.0 ===Use=== module add hashrf-6.0.0 hashrf ===Documentation=== https://code.google.com/p/hashrf/ ===Licence=== GNU GPL v3 ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== https://code.google.com/p/hashrf/ [[Kategorie:Applications]] sp77y76ltp9bla6zlachzsd19djsv8t 0 267 4863 4855 2016-05-13T09:09:31Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--SRA Toolkit|Česká verze--> =Description= =Availability= Version 2.3.2 =Licence= Freely available - unlicensed. =Use= You can load the module via the command module add sratools-2.3.2 This makes available the path to the program binaries and also the shared libraries. ===Usage mistakes=== ====name not found while resolving ...==== If you try to use the tools like: cd $SCRATCHDIR cp -r /storage/brno2/home/$user/data/* . fastq-dump --split-3 FILE.sra > out.fastq then occurs the error: 2013-04-16T12:54:30 fastq-dump.2.3.2 err: libs/vfs/resolver.c:790:VResolverAlgRemoteResolve: name not found while resolving tree within virtual file system module - failed to open 'FILE.sra' '''Solution:''' The tools do not count with the current directory path. So you can repair the computation simply by adding the path to your file: fastq-dump --split-3 ./FILE.sra > out.fastq =Documentation= http://0-www.ncbi.nlm.nih.gov.elis.tmu.edu.tw/Traces/sra/sra.cgi?view=software =Program manager= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://0-www.ncbi.nlm.nih.gov.elis.tmu.edu.tw/Traces/sra/sra.cgi?view=software [[Category:Applications]] [[Category:English]] bvviqcnhb0uvm3fq0m4vgv1p7wfkrtn 0 258 4868 4847 2016-05-13T09:09:34Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Octave|Česká verze--> ===Description=== GNU Octave is a high-level language, primarily intended for numerical computations. It provides a convenient command line interface for solving linear and nonlinear problems numerically, and for performing other numerical experiments using a language that is mostly compatible with Matlab. It may also be used as a batch-oriented language. Octave has extensive tools for solving common numerical linear algebra problems, finding the roots of nonlinear equations, integrating ordinary functions, manipulating polynomials, and integrating ordinary differential and differential-algebraic equations. It is easily extensible and customizable via user-defined functions written in Octave's own language, or using dynamically loaded modules written in C++, C, Fortran, or other languages. ===Availability=== '''Version 3.8.2''' (module <code>octave-3.8.2</code>) -- compiled by GNU compilers with the usage of Intel MKL libraries (BLAS, LAPACK, FFTW). ===Use=== '''Command-line version''': module add octave-3.8.2 octave '''Graphical version''' -- start a remote desktop following the [[Remote_desktop|steps described here]] and run: module add octave-3.8.2 octave-gui ===Documentation=== http://www.gnu.org/software/octave/support.html ===Licence=== GNU General Public License ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.gnu.org/software/octave/ [[Kategorie:Applications]] m1rr3yhzksel387w3ufuea08uw9but1 0 279 4875 4794 2016-05-13T09:09:46Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Multidis|Česká verze--> ===Description=== '''Multidis''' is ... ===Availability=== '''Module <code>multidis</code>:''' ''Multidis version ...'', 64-bit, compiled by Intel compiler with MPI and Intel MKL libraries support ===Use=== ... ===Documentation=== http://moldyn.vsb.cz/index.php/software-menu/2-uncategorised/5-multidyn ===Licence=== open source ===Supported platforms=== amd64 ===Program administrator=== Martin Stachoň (VŠB-TU), Ivan Janeček (UGN CAS), Tom Rebok [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://moldyn.vsb.cz/ [[Kategorie:Applications]] n43smtahvvri39p9fek6f2yk54dp9k4 0 180 4876 4730 2016-05-13T09:09:47Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Grace|Česká verze--> =Description= Grace is a WYSIWYG 2D plotting tool for the X Window System and M*tif. There are some features of this program mentioned: *WYSIWYG design *Convenient point-and-click graphical user interface *Precise control of graph features *Exports vector graphics to (E)PS, PDF, MIF, and SVG formats *Supports cross-platform PNG, PNM, and JPEG formats *Device-independent Type1 font rastering, including anti-aliasing, ligatures, and kerning *Unlimited number of graphs *Unlimited number of curves on a graph *etc... =Availability= Versions '''5.1.23''' and 5.1.22 are available. =Use= Load module ("module ad grace") and run needed program (convcal, fdf2fit, gracebat, grconvert, xmgrace). {{Application with GUI}} =Documentation= [http://plasma-gate.weizmann.ac.il/Grace/doc/UsersGuide.html On-line] docuemntation is available. =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= URL: http://plasma-gate.weizmann.ac.il/Grace/ [[Category:Applications]] [[Category:English]] 321mkqph5v8rz20slrag8pirgxkq3f6 0 390 4878 4770 2016-05-13T09:09:49Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--PALM|Česká verze--> =Description= PALM is a large-eddy simulation (LES) model for atmospheric and oceanic flows which is especially designed for performing on massively parallel computer architectures. Some of PALM's highlights are: * excellent scaling, so far tested up to 32000 cores * online data analysis (during model runs) in order to avoid I/O bottlenecks * topography realized on cartesian grid (allows for steep topography) * non-cyclic horizontal boundary conditions including turbulent inflow * code can be switched to ocean version with salinity equation and equation of state for seawater * embedded parallelized Lagrangian particle model for various applications (footprint calculation, simulation of cloud droplet growth, visualization, etc.) * interface which allows users to plug in their own code extensions without modifying the default code * advanced shell scripts for installing and running the code in interactive and batch mode are available * code is permanently maintained and improved by the PALM group and other users; code management is based on subversion =Availability= Program is actually available from 2015/08/13 SVN version. Module versions: palm-20150813 =Licence= [http://www.gnu.org/licenses/gpl-3.0.en.html GNU/GPL licence] =Use= Initialize environment by command module add palm-20150813 =Documentation= Documentation is available at http://palm.muk.uni-hannover.de/wiki/TracGuide . =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://palm.muk.uni-hannover.de/ [[Category:Applications]] [[Category:English]] pg5m9wtfj2swsc43ylsmg215359ik8m 0 171 4881 4762 2016-05-13T09:09:54Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Structure|Česká verze--> ===Description=== The program structure is a free software package for using multi-locus genotype data to investigate population structure. Its uses include inferring the presence of distinct populations, assigning individuals to populations, studying hybrid zones, identifying migrants and admixed individuals, and estimating population allele frequencies in situations where many individuals are migrants or admixed. It can be applied to most of the commonly-used genetic markers, including SNPS, microsatellites, RFLPs and AFLPs. ===Availability=== Program is available in version 2.2.3 and 2.3.3 ===Use=== '''Version 2.3.3:''' Initialize the enviroment ("module add structure-2.3.3") and run "structure". '''Version 2.2.3:''' Initialize the enviroment ("module add jdk-1.6.0 structure-2.2.3") and run "structure.exe". ===Documentation=== '''Version 2.3.3:''' Documentation is available at [http://pritch.bsd.uchicago.edu/structure_software/release_versions/v2.3.3/html/structure.html] . '''Version 2.2.3:''' Documentation is available at [http://pritch.bsd.uchicago.edu/software/structure2_2.html] . '''Note''': There are number of program parameters that are set by the user and stored in two files (''mainparams'' and ''extraparams''), which are read every time the program executes. Both the files must be in a directory from where is the Structure executed. See http://pritchardlab.stanford.edu/software/structure_v.2.3.1/documentation.pdf (chapter 7, page 20) for more dotails. ===Program administrator=== Tom Rebok ([mailto:meta@cesnet.cz meta@cesnet.cz]) ===Homepage=== URL: http://pritch.bsd.uchicago.edu/structure.html ===Parallelisation using R library ParallelStructure=== On the basis of very good inspiration I described way how to launch more Structure runs in parallel using R script. See, please, https://trapa.cz/en/structure-r-linux You can easily launch for example 100 Structure runs on 10 CPUs. [[Kategorie:Applications]] l6wz5xpx1fkxf83qobnu6267mg2yps1 0 384 4882 4751 2016-05-13T09:09:54Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--QUAST|Česká verze--> = Popis = QUAST performs fast and convenient quality evaluation and comparison of genome assemblies. QUAST computes a number of well-known metrics, including contig accuracy, number of genes discovered, N50, and others, as well as introducing new ones, like NA50 (see details in the paper andin the manual). A comprehensive analysis results in summary tables (in plain text, tab-separated and LaTeX formats) and colorful plots. The tool also produces web-based reports condensing all information in one easy to navigate file. QUAST has an intuitive command-line interface and a detailed manual to help users run it and understand its output. = Availability= Application is available for all usrs of MetaCentrum. = Use= Load module: module add quast-2.3 Run the application by command <code>quast.py</code>, e.g quast.py test_data/contigs_1.fasta \ test_data/contigs_2.fasta \ -R test_data/reference.fasta.gz \ -G test_data/genes.txt \ -O test_data/operons.txt \ --threads 4 You can find files used in the example above at sourceforge.net/projects/quast/ For appropriate use of parallelization is important to set the parameter <code>threads</code>, which specified number of used computational threads. It is set to the number of CPU cores implicitly. Favourable solution is use for this purpose the variable which represents number of reserved CPU cores: --threads $PBS_NUM_PPN = Documentation = On-line documentation you can find on http://quast.bioinf.spbau.ru/manual.html = Licence= GNU General Public License version 2.0 (GPLv2) = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://bioinf.spbau.ru/quast [[Category:Applications]] [[Category:English]] oafg4e1d4mykvb0qmtzcw45yhbzusc2 0 182 4884 4828 2016-05-13T09:09:58Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Wine|Česká verze--> ===Description=== Wine is a translation layer (a program loader) capable of running Windows applications on Linux and other POSIX compatible operating systems. Windows programs running in Wine act as native programs would, running without the performance or memory usage penalties of an emulator, with a similar look and feel to other applications on your desktop. ===Availability=== Program is actually available in version Wine-1.0.1 stable and Wine-1.1.16. ===Use=== Load module ("module ad wine" for version 1.0.1) or ("module ad wine-1.1.16") and run program. ===Documentation=== [http://winehq.org/documentation/ On-line] documentation is available. ===Supported platforms=== Linux ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://www.winehq.org/ [[Kategorie:Applications]] 6hx8jmkoa78ckpl0aieq9f97bdqz1wa 0 202 4894 4747 2016-05-13T09:10:06Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--ABySS|ABySS--> ===Description=== ABySS is a de novo, parallel, paired-end sequence assembler that is designed for short reads. The single-processor version is useful for assembling genomes up to 100 Mbases in size. The parallel version is implemented using MPI and is capable of assembling larger genomes. ===Availability=== Program is available in version 1.9.0 ===Use=== Initialize enviroment ("module add abyss-1.9.0") and run "abys-pe". Or run "abyss-pe np=8" for 8 thread version, or "mpirun -np 8 ABYSS-P" from MPI version (add openmpi modul too). ===Documentation=== http://www.bcgsc.ca/downloads/abyss/doc ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.bcgsc.ca/platform/bioinfo/software/abyss [[Kategorie:Applications]] ipp1hby264v1p515uxl7pslgxa7cr5n 0 250 4909 4797 2016-05-13T09:10:23Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Picard|Česká verze--> = Description = Picard tools is a set of tools (in Java) for working with next generation sequencing data in the BAM format. = Availability = Version 1.80 and 1.100. Freely available to users. Available modules: picard-1.80 picard-1.100 = Licence = Apache v 2.0, MIT licence. = Use = Example of environment initialization: module add picard-1.100 Initialization makes available also module java 1.6.0 and system variable $PICARD pointing into Picard Tools install dir. Usage of one of the tools with sample data: java -jar $PICARD/CreateSequenceDictionary.jar REFERENCE=$PICARD/test/pokusni.fasta OUTPUT=~/out.fasta = Documentation = Dokumentation is available at http://picard.sourceforge.net/command-line-overview.shtml, FAQ can be found on wiki of Picard Tools project on [http://sourceforge.net/apps/mediawiki/picard/index.php?title=Main_Page SourceForge]. = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://picard.sourceforge.net/ and http://sourceforge.net/projects/picard/ [[Category:Applications]] [[Category:English]] ijg2map1flolzwzeguykfvkviwcd6v9 0 306 4912 4744 2016-05-13T09:10:25Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Clview|Česká verze--> = Description = This is a graphical, interactive tool for inspecting the ACE format assembly files generated by CAP3 or phrap. Beasides the ACE files, the program also supports a custom cluster layout format for the overview of a possible multiple alignments generated just from pairwise alignments, where no detailed nucleotide level alignment is needed and provided. The "containment clustering" program (nrcl) mentioned in the TGI Clustering tools(TGICL) above can generate such a "cluster layout" file (*.lyt). = Availability = Version 2010. Freely available for every user. = Licence = [http://compbio.dfci.harvard.edu/tgi/software/artistic.html Free Artistic Licence] = Use = Initialize module with command: module add clview-2010 {{Application with GUI}} = Documentation = No documentation available. = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://compbio.dfci.harvard.edu/tgi/software/ [[Category:Applications]] [[Category:English]] o2i26ijcr0hiwjjdnomr7z7bcwgt801 0 192 4915 4763 2016-05-13T09:10:29Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--OpenCV|Česká verze--> ===Description=== OpenCV module contains C++ and Python libraries (.h header files, static and shared libraries, Python module) for image processing. You can read more OpenCV in documentation (see link below). Currently, OpenCV is compiled without GPU support (Cuda). ===Availability=== Current and preferred OpenCV version in MetaCentrum environment: OpenCV 2.4.8 is available in '''opencv-2.4''' module. Older version 2.3.2 (trunk version), available in '''opencv-2.3.2''' module. ===Use: Python=== OpenCV can be used as a Python library. This module is compiled with Python 2.7 and Numpy support. To initialize your environment, use: <source lang="text"> # for opencv-2.4 module (preferred) module add python-2.7.5 numpy-1.7.1-py2.7 ffmpeg opencv-2.4 # for older opencv-2.3.2 module module add python-2.6.2 numpy-py2.6 ffmpeg opencv-2.3.2 </source> Then, in Python, library (Python module) cv2 is available, you can import it and use it: <source lang="python"> import cv2 img = cv2.imread('test.png') </source> ===Use: C++=== Here is a short demonstration, how to compile a C++ project with OpenCV. OpenCV module uses [[ffmpeg_application]] module internally for video processing. Initialize your environment: <source lang="text"> module add cmake-2.8 ffmpeg opencv-2.4 </source> Module cmake is used for c++ project compilation. Our sample project is available here: [http://www.kky.zcu.cz/public/metacentrum_opencv_demo.zip metacentrum_opencv_demo.zip]. Decompress this ZIP file in your home directory, at any metacentrum frontend machine. Directory "metacentrum_opencv_demo" contains source cpp file, CMakeLists.txt for cmake, and several sample videos in "samples" subdirectory. Create new directory "metacentrum_opencv_demo_build" (as a sibling directory to "metacentrum_opencv_demo" directory). Here, we will make a compilation and build of whole project. Similarly, create new directory "metacentrum_opencv_demo_install", where resulting binary file will be installed. In "metacentrum_opencv_demo_build" directory, execute: <source lang="bash"> ccmake ../metacentrum_opencv_demo </source> This will launch cmake GUI and create new configuration for our build. Configure the project as follows: <source lang="text"> CMAKE_BUILD_TYPE CMAKE_INSTALL_PREFIX /home/campr/metacentrum_opencv_demo_install OpenCV_DIR /afs/ics.muni.cz/software/opencv-2.4/share/OpenCV </source> OpenCV_DIR should be set automatically. Adjust CMAKE_INSTALL_PREFIX to link to your "metacentrum_opencv_demo_install" directory. Press "c" to configure, "g" to generate project (makefile etc. will be created). In "metacentrum_opencv_demo_build" directory execute: <source lang="bash"> make install </source> This will compile the project. In case no error was thrown, the resulting files (binary executable and sample videos) will be installed to "metacentrum_opencv_demo_install" directory. To test, in "metacentrum_opencv_demo_install/bin" directory, execute: <source lang="bash"> ./opencv_demo ../samples/test.avi </source> This will output a set of points (one point per frame from input video). ===Documentation=== [http://opencv.itseez.com/ Official on-line OpenCV documentation] ===Licence=== BSD license (free for both academic and commercial use) ===Supported platforms=== Linux (in metacentrum environment), generally works with Windows and MacOS too ===Program manager=== [http://www.kky.zcu.cz/en/people/campr-pavel Pavel Campr] ===Homepage=== [http://opencv.willowgarage.com OpenCV homepage] [[Kategorie:Applications]] ihp1rxhlodawhuchj0utw4eyra75csr 0 365 4916 4742 2016-05-13T09:10:30Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--OMA|Česká verze--> = Description = The OMA algorithm uses as input the sequences from complete genomes and yields orthologous relations and groups of orthologs. The process works in 4 phases: * All-against-all, the pairwise alignments between every pair of proteins of all genomes (“all-against-all”). For significant ones, an evolutionary distance is estimated in PAM units.2. * Formation of “stable pairs” (SPs), the mutually closest pairs within a confidence interval, to avoid exclusion of n:m orthologs. * Detection of paralogs among SPs using witnesses of non-orthology [2]. Pairs surviving this step are named “verified pairs” (VPs). At this point, they are very likely to be orthologs. * Formation of OMA groups, made of a subset of proteins in which all pairs are VPs (clique). = Availability = Version 0.99y. Freely available to users. Modules: oma-0.99y = Licence = [https://www.mozilla.org/MPL/2.0/ Mozilla Public License Version 2.0] = Use = Initialize environment with command: module add oma-0.99y = Documentation = Basic documentation is at http://omabrowser.org/standalone/ = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://omabrowser.org [[Category:Applications]] [[Category:English]] nlim84hk7clqnc3hgpdrvptrbes2yv6 0 282 4924 4813 2016-05-13T09:10:48Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Scikit-learn|Česká verze--> ===Description=== '''Scikit-learn''' is a Python language extension that uses Numpy and Scipy to provide simple and efficient tools for data mining and data analysis. ===Availability=== * '''Scikit-learn version 0.14.1:''' compiled with Intel MKL 11.0 '''(internally parallelized)''', which results in '''vastly improved computation performance''' ** module <code>sklearn-0.14.1-py2.7</code> * ''Scikit-learn version 0.11:'' compiled with Intel MKL 11.0 '''(internally parallelized)''', which results in '''vastly improved computation performance''' ** module <code>sklearn-0.11-py2.6</code> ===Use=== Load module (<code>module ad "module_name"</code>), then you are allowed to import ''Scikit-learn'' libraries into Python. ===Documentation=== [http://scikit-learn.org/stable/documentation.html On-line] documentation is available. ===Supported platforms=== Linux ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://scikit-learn.org/stable/index.html [[Kategorie:Applications]] qqusd2s67ta1mo97xidi30fgc2hlzca 0 367 4933 4802 2016-05-13T09:10:57Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--SPIDER|Česká verze--> = Description = SPIDER (System for Processing Image Data from Electron microscopy and Related fields) is an image processing system for electron microscopy. = Availability = Versions 21.13 and 22.03. Modules available: spider-21.13-mpi-intel spider-22.03-mpi-intel = Licence = [http://www.gnu.org/licenses/gpl-2.0.html GPLv2] = Use = module add spider-22.03-mpi-intel spider = Documentation = http://spider.wadsworth.org/spider_doc/spider/docs/documents.html = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://spider.wadsworth.org/spider_doc/spider/docs/spider.html [[Category:Applications]] [[Category:English]] th3iv7hw1wu9cotvl4n917d9jfnnh81 0 288 4934 4799 2016-05-13T09:10:58Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--FPC|Česká verze--> =Description= FPC (FingerPrinted Contigs) takes as input a set of clones and their restriction fragments (called Bands) and assembles the clones into contigs. =Availability= Program is actually freely available to all users in version 9.4. Module versions: fpc-9.4 =Licence= Unknown, but there is a citation request: ''In any publication or poster that refers to fingerprinted contigs built with FPC, you will reference one of the following papers: Soderlund et al. 1997, Soderlund et al. 2000, Engler et al. 2003. The full references for these papers are on the previous [http://www.agcol.arizona.edu/software/fpc/#ref page]. If you refer to FPC from a web site, you will refer to the URL www.agcol.arizona.edu/software/fpc.'' =Use= Initialize environment by command module add fpc-9.4 This makes available the path to program binary. There are also available tutorials in <tt>/software/fpc-9.4/tutorials</tt> directory. {{Application with GUI}} =Documentation= User documentation is available in file <tt>/software/fpc-9.4/doc/FPChelpdoc.htm</tt> and at http://www.agcol.arizona.edu/software/fpc/#fpc webpage. =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://www.agcol.arizona.edu/software/fpc [[Category:Applications]] [[Category:English]] 2rkinqyzv7019alxe490787wk9qrvf4 0 174 4937 4840 2016-05-13T09:11:03Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--R|Česká verze--> =Description= R is a language and environment for statistical computing and graphics. R provides a wide variety of statistical (linear and nonlinear modelling, classical statistical tests, time-series analysis, classification, clustering, ...) and graphical techniques, and is highly extensible. R is an integrated suite of software facilities for data manipulation, calculation and graphical display. R includes *an effective data handling and storage facility, *a suite of operators for calculations on arrays, in particular matrices, *a large, coherent, integrated collection of intermediate tools for data analysis, *graphical facilities for data analysis and display either on-screen or on hardcopy and printed version. *a well-developed, simple and effective programming language which includes conditionals, loops, user-defined recursive functions and input and output facilities. =Availability= * '''Module <code>R-3.2.3-intel</code>:''' ''R version 3.2.3'' with blas, lapack and tcltk support * Module <code>R-3.1.1</code>: ''R version 3.1.1'' with blas, lapack and tcltk support * Module <code>R-3.1.0</code>: ''R version 3.1.0'' with blas and lapack support * Module <code>R-3.0.1</code>: ''R version 3.0.1'' with blas and lapack support * Module <code>R-2.14.0</code>: ''R version 2.14.0'' * Module <code>R-2.8.1</code>: ''R version 2.8.1'' =Use= Initialize enviroment ("module add R" for 2.14.0 version) and run "R". For batch regime, use your R file (commands.R) with commands for R R CMD BATCH commands.R {{Application with GUI}} ===Using Rmpi=== How to use the Rmpi package? At first prepare a file hello_world_rmpi.R which will contain the following example (adopted from http://math.acadiau.ca/ACMMaC/Rmpi/sample.html): # Load the R MPI package if it is not already loaded. if (!is.loaded("mpi_initialize")) { library("Rmpi") } # Spawn as many slaves as possible mpi.spawn.Rslaves() # In case R exits unexpectedly, have it automatically clean up # resources taken up by Rmpi (slaves, memory, etc...) .Last <- function(){ if (is.loaded("mpi_initialize")){ if (mpi.comm.size(1) > 0){ print("Please use mpi.close.Rslaves() to close slaves.") mpi.close.Rslaves() } print("Please use mpi.quit() to quit R") .Call("mpi_finalize") } } # Tell all slaves to return a message identifying themselves mpi.remote.exec(paste("I am",mpi.comm.rank(),"of",mpi.comm.size())) # Tell all slaves to close down, and exit the program mpi.close.Rslaves() mpi.quit() Add the openmpi modul and run the batch in "R" module add R openmpi-1.6.5-intel export OMPI_MCA_btl=sm,tcp,self export OMPI_UNIVERSE_SIZE=`expr $TORQUE_RESC_TOTAL_PROCS - 1` mpirun R CMD BATCH hello_world_rmpi.R {{MPI application}} ===UTF-8 support=== This command sets the support for utf-8 at the beginning of the R script. For example where you need to import the Czech diacritics by function read.csv(): Sys.setlocale(category = "LC_ALL", locale = "en_US.UTF8") =Documentation= Documentation is available on [http://cran.r-project.org/manuals.html WWW server] or locally in program directory. =Supported platforms= Different versions of Unixu, Windows, MacOS. =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= URL: http://www.r-project.org/ =FAQ (usefull links)= * Supercomputing with R: http://cran.r-project.org/web/views/HighPerformanceComputing.html * Instalation of package into storage directory /storage/brno2/home/LOGNAME/Rpackages/ (change LOGNAME by your login and create Rpackages directory before running following commands) from R console <source lang="bash"> install.packages("PACKAGE_NAME",lib="/storage/brno2/home/LOGNAME/Rpackages/") </source> loading of such package <source lang="bash"> library(PACKAGE_NAME,lib.loc="/storage/brno2/home/LOGNAME/Rpackages/") </source> To set your library location you can set R_LIBS variable to existing folder <source lang="bash"> mkdir /storage/brno2/home/$LOGNAME/Rpackages/ export R_LIBS="/storage/brno2/home/$LOGNAME/Rpackages" R > .libPaths() [1] "/auto/brno2/home/$LOGNAME/Rpackages" [2] "/afs/ics.muni.cz/software/R-3.1.0/lib/R/library" > install.packages("PACKAGENAME") </source> and for bioconductor <source lang="bash"> > source("http://bioconductor.org/biocLite.R") > biocLite("PACKAGENAME") </source> * simple one core R script <source lang="bash"> #PBS -N helloR #PBS -l walltime=2h #PBS -l nodes=1:ppn=1 #PBS -l mem=15gb #PBS -l scratch=1gb #clean scratch after the end trap 'clean_scratch' TERM EXIT #INPUT directory - with R script and other inputs MYIN=/storage/brno2/home/xfibich/helloR #INPUT R script MYR=hello.R #OUTPUT directory - where to store results MYOUT=/storage/brno2/home/xfibich/helloRout mkdir -p $MYOUT module add R-3.1.1 # go to scratch directory cd $SCRATCHDIR || exit 2 # copy all files from $MYIN cp $MYIN/* . R -q --vanilla < $MYR > $MYR.output #copy everything to MYOUT cp * $MYOUT/ </source> script is send for computation by command <source lang="bash"> qsub helloR.sh </source> [[Category:Applications]] [[Category:English]] go9ehlfwseweatojraewtoo42au8bfr 0 315 4944 4871 2016-05-13T09:11:10Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--CM5PAC|Česká verze--> = Description = '''CM5PAC''' is a package to carry out a calculation of CM5 partial atomic charges using Hirshfeld atomic charges from Gaussian 09's output file. The CM5 charges are based on Charge Model 5 (CM5) that yields class IV partial atomic charges by mapping from those obtained by Hirshfeld population analysis of density functional electronic charge distributions. To run a CM5 calculation CM5PAC employs Gaussian 09's output file containing a set of Hirshfeld atomic charges. '''''Warning:''' CM5 charges can be computed along with Hirshfeld charges using Revision D.01 of Gaussian 09 without CM5PAC but such calculations may produce wrong CM5 charges in certain cases due to a bug which will be fixed in the next revision of Gaussian. In the meantime, we strongly recommend to use CM5PAC to calculate CM5 charges because CM5PAC does not experience such a problem. '' = Availability = Version 2013 is available. = License = Free to use, requires citation. = Use = Once a module is loaded, the <code>cm5pac</code> application may be used. As an input, it requires Gaussian 09's output file, where the results of the CM5PAC are also written: <source lang=bash> $ module add cm5pac-2013 $ cm5pac G09calc.log </source> = Documentation = http://comp.chem.umn.edu/cm5pac/cm5pac_Manual_v2013.pdf = Program administrator = Tom Rebok ([mailto:meta@cesnet.cz meta@cesnet.cz]) = Homepage = http://comp.chem.umn.edu/cm5pac/ [[Kategorie:Applications]] [[Kategorie:English]] q02jtp2hozi86qeyux1eph57kxit2bo 0 245 4955 4788 2016-05-13T09:11:21Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Mosaik|Česká verze--> ===Popis=== '''MOSAIK''' is a reference-guided assembler comprising of four main modular programs: * ''MosaikBuild'' -- converts various sequence formats into Mosaik’s native read format * ''MosaikAligner'' -- pairwise aligns each read to a specified series of reference sequences * ''MosaikSort'' -- resolves paired-end reads and sorts the alignments by the reference sequence coordinates * ''MosaikAssembler'' -- parses the sorted alignment archive and produces a multiple sequence alignment which is then saved into an assembly file format ===Dostupnost=== * '''Module <code>mosaik-1.1</code>:''' ''Mosaik version 1.1'', 64-bit * '''Module <code>mosaik-2.1</code>:''' ''Mosaik version 2.1'', 64-bit ===Use=== '''1. Running the application -- ''Batch mode:''''' * prepare the job description script -- use a general skeleton supplemented by the following lines: <source lang="bash"> ... # load the application module module add mosaik-2.1 # run the computation MosaikAligner -in <filename> -out <filename> -ia <filename> ... </source> * pass the job description file to the scheduler together with (at least) the requested number of processors and requested amount of memory <source lang="bash"> $ qsub -l nodes=1:ppn=X -l mem=Yg mydescriptionfile.sh </source> '''2. Running the application -- ''Interactive mode:''''' * ask the scheduler for an interactive job having a single node and a desired number of processors (<code>ppn</code> attribute) reserved <source lang="bash"> $ qsub -I -l nodes=1:ppn=X -l mem=Yg </source> :''Note: Do not forget to apppropriately set the amount of requested memory (<code>mem</code> attribute) and/or another job requirements'' * load the Mosaik module <source lang="bash"> $ module add mosaik-2.1 </source> * prepare the input files and run the computation <source lang="bash"> $ MosaikAligner -in <filename> -out <filename> -ia <filename> </source> ===Documentation=== * version 1.1: [http://mosaik-aligner.googlecode.com/files/Mosaik%201.0%20Documentation.pdf producer's webpage] or locally in the program directory (<code>/software/mosaik/1.1/do</code>) * version 2.1: [http://code.google.com/p/mosaik-aligner/ Google code] ===License=== open-source under the terms of the [http://www.gnu.org/licenses/gpl-2.0.html GPL 2.0+ license] and the [http://opensource.org/licenses/mit-license.php MIT licence] ===Supported platforms=== amd64 ===Program administrator=== Tom Rebok [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== * http://bioinformatics.bc.edu/marthlab/Mosaik * http://code.google.com/p/mosaik-aligner/ [[Kategorie:Applications]] ejmwqy2ruwbrgw3pdgufbehtv1bzapx 0 185 4958 4739 2016-05-13T09:11:23Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Numpy|Česká verze--> =Description= Numpy is a Python language extension that defines the numerical array and matrix type and basic operations over them. =Availability= There are several versions of Numpy in Metacentrum. In addition to later described compilations is Numpy available in modules python26-modules-* and python27-modules-*. See page [[Python - modules]]. * '''Numpy version 1.8.0:''' compiled with [[INTEL CDK application|Intel CDK 14]] '''(internally parallelized)''', which results in '''vastly improved computation performance''' ** modules <code></code> and <code></code> * Numpy version 1.7.1: compiled with Intel MKL 11.0 '''(internally parallelized)''', which results in '''vastly improved computation performance''' ** modules <code>numpy-1.7.1-py2.7</code> *** prepared to be used with Python 2.7 (module ''*-py2.7'') * ''Numpy version 1.6.2:'' compiled with Intel MKL 11.0 '''(internally parallelized)''', which results in '''vastly improved computation performance''' ** modules <code>numpy-1.6.2-py2.5</code> and <code>numpy-1.6.2-py2.6</code> *** prepared to be used both with Python 2.5 (module ''*-py2.5'') as well as Python 2.6 (module ''*-py2.6'') * ''Numpy version 1.6.0 beta 1:'' compiled with standard BLAS libraries ** modules <code>numpy-1.6.0b1-py2.5</code> and <code>numpy-1.6.0b1-py2.6</code> *** prepared to be used both with Python 2.5 (module ''*-py2.5'') as well as Python 2.6 (module ''*-py2.6'') =Use= Load module (<code>module ad 'module_name'</code>), then you can import numpy libraries into Python (<code>import numpy</code>) or run the program f2py, by command <code>f2py</code> with neccessary parameters (see: <code>man f2py</code>). There is a test program available for download [[Media:numpytest.zip|HERE]]. '''''Note:''' The modules version 1.6.0 do not automatically load the Python modules -- if you need them, load them manually.'' =Documentation= [http://docs.scipy.org/doc/ On-line] documentation is available. =Supported platforms= Linux =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= URL: http://www.scipy.org/Getting_Started [[Kategorie:Applications]] [[Kategorie:English]] kzj3jnpep741x5l6hzexrzh3gyl9ozm 0 149 4960 4740 2016-05-13T09:11:26Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Gamess|Česká verze--> ===Description=== GAMESS is a program for ab initio molecular quantum chemistry. Briefly, GAMESS can compute SCF wavefunctions ranging from RHF, ROHF, UHF, GVB, and MCSCF. Correlation corrections to these SCF wavefunctions include Configuration Interaction, second order perturbation Theory, and Coupled-Cluster approaches, as well as the Density Functional Theory approximation. ===Availability=== Version GAMESS US standard version (April 2008) without additional modules SIMO/MM, VB and NEO, but with supportive graphical utilities is in module gamess-us. Actual version (June 2011) is available in module "gamess-us-Jun11" only in 64 bit version with MPI support. ===Use=== Load module ("module ad gamess-us" or gamess-us-Jun11), licence is free, but every research group have to register at http://www.msg.chem.iastate.edu/gamess/License_Agreement.html (After resending us the answer to your registration, you will be allowed to use module.) Run script rungms with parameter name of input to run module. The module sets implicitly variable SCR to $SCRATCHDIR, so if you want to use different value, set it in the script of the task (and create directory, if it doesnt exist yet) before running rungms. ===Documentation=== Original documentation you can find in /software/gamess-us-Apr08/doc. Online version: http://www.msg.chem.iastate.edu/gamess/documentation.html ===Supported platforms=== Linux ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.msg.chem.iastate.edu/gamess/ ===Example=== Copy input file for gamess "pokus.inp" (example file) into /home/poli29 (example of your home directory). Then just run "rungms" with one parameter – name of input. May you have problem with not having your own subdirectory in skirit frontend in /scratch as you have in the normal nodes. So in this case you have to set enviroment variable SCR in shell to an existing directory (gamess leave file *.dat (PUNCH) into it) . <source lang="bash"> $ module add gamess-us $ cp /afs/ics.muni.cz/software/gamess-us-Apr08/tests/exam01.inp . $ rungms exam01.inp >exam01.log 2>exam01.err $ tail -20 exam01.log TOTAL WALL CLOCK TIME= 0.0 SECONDS, CPU UTILIZATION IS 100.00% 440000 WORDS OF DYNAMIC MEMORY USED EXECUTION OF GAMESS TERMINATED NORMALLY Mon Jan 19 10:09:25 2009 DDI: 262808 bytes (0.3 MB / 0 MWords) used by master data server. ---------------------------------------- CPU timing information for all processes ======================================== 0: 0.140008 + 0.024001 = 0.164009 1: 0.000000 + 0.004000 = 0.004000 ---------------------------------------- ddikick.x: exited gracefully. ----- accounting info ----- Mon Jan 19 10:09:28 CET 2009 Files used on the master node skirit.ics.muni.cz were: -rw-r--r-- 1 salvet users 1136 Jan 19 10:09 /scratch/salvet/job_12345.arien.ics.muni.cz/exam01.F05 -rw-r--r-- 1 salvet users 180020 Jan 19 10:09 /scratch/salvet/job_12345.arien.ics.muni.cz/exam01.F08 -rw-r--r-- 1 salvet users 1308928 Jan 19 10:09 /scratch/salvet/job_12345.arien.ics.muni.cz/exam01.F10 -rw-r--r-- 1 salvet users 14141 Jan 19 10:09 /scratch/salvet/job_12345.arien.ics.muni.cz/exam01.dat 0.0u 0.0s 0:05.56 1.2% 0+0k 0+0io 0pf+0w </source> [[Kategorie:Applications]] p2lrd45k15k0xiburpqij6u5f7iava0 0 166 4962 4758 2016-05-13T09:11:29Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--PhyloBayes|Česká verze--> =Description= PhyloBayes is a Bayesian Monte Carlo Markov Chain (MCMC) sampler for phylogenetic reconstruction using protein alignments. Compared to other phylogenetic MCMC samplers, the main distinguishing feature of PhyloBayes is the underlying probabilistic model, CAT (Lartillot and Philippe, 2004). CAT is a mixture model especially devised to account for site-specific features of protein evolution. It is particularly well suited for large multigene alignments, such as those used in phylogenomics. =Availability= Program is available in version 3.2f, 3.3b and MPI 1.5a. Usable modules: phylobayes-3.2f phylobayes-3.3b phylobayes-mpi-1.5a =Use= ===Normal version=== Initialize environment with command ''module add''. Example: module add phylobayes-3.3b and run command ''pb''. ===Parallel version=== Initialize environment with command ''module add''. Example: module add phylobayes-mpi-1.5a and run ''pb_mpi''. The version numbers differs in parallel application, so version 1.5a is the last in May 2014. {{MPI application}} =Documentation= Documentation is available on [http://megasun.bch.umontreal.ca/People/lartillot/www/index.htm WWW server]. =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= URL: http://megasun.bch.umontreal.ca/People/lartillot/www/index.htm [[Category:Applications]] [[Category:English]] 9d9oym2t4724i5hv9htfppp2d2aux1k 0 248 4969 4929 2016-05-13T09:11:40Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--OpenFst|Česká verze--> ===Description=== '''OpenFst''' is a library for constructing, combining, optimizing, and searching weighted ''finite-state transducers (FSTs)''. Weighted finite-state transducers are automata where each transition has an input label, an output label, and a weight. The more familiar finite-state acceptor is represented as a transducer with each transition's input and output label equal. Finite-state acceptors are used to represent sets of strings (specifically, regular or rational sets); finite-state transducers are used to represent binary relations between pairs of strings (specifically, rational transductions). The weights can be used to represent the cost of taking a particular transition. FSTs have key applications in speech recognition and synthesis, machine translation, optical character recognition, pattern matching, string processing, machine learning, information extraction and retrieval among others. Often a weighted transducer is used to represent a probabilistic model (e.g., an n-gram model, pronunciation model). FSTs can be optimized by determinization and minimization, models can be applied to hypothesis sets (also represented as automata) or cascaded by finite-state composition, and the best results can be selected by shortest-path algorithms. ===Availability=== * '''Module <code>openfst-1.1</code>:''' ''OpenFst version 1.1'', 64-bit, ... * '''Module <code>openfst-1.2</code>:''' ''OpenFst version 1.2'', 64-bit, ... ===Use=== '''Running the application''' * ask for interactive task with one node and specified processors count (the <code>ppn</code> parameter) <source lang="bash"> $ qsub -I -l nodes=1:ppn=X -l mem=Yg </source> Also specify the memory size (the <code>mem</code> parameter) or any other task requirements. * call the application module into your system <source lang="bash"> $ module add openfst-1.1 </source> * Then simply run any command from OpenFST library with desired parameters. When you run any command with switch --help, will obtain simple manual: <source lang="bash"> $ fstcompose --help Composes two FSTs. Usage: fstcompose in1.fst in2.fst [out.fst] Flags: connect filter Flags Description: --acceptor: type = bool, default = false Input in acceptor format ...atd. </source> * Concatenation of some commands with use of the pipe brings an advantage to your work: <source lang="bash"> $ fstcompose <in1.fst <in2.fst | fstrmepsilon | fstminimize | fstdeterminize >out.fst </source> ===Documentation=== * [http://www.openfst.org/twiki/bin/view/FST/WebHome online at producer's webpage] ===License=== open-source, distributed under the terms of the [http://www.apache.org/licenses/LICENSE-2.0 Apache license]. ===Supported platforms=== amd64 ===Program administrator=== Jan Vavruška (vavruska (at) kky.zcu.cz) ===Homepage=== http://www.openfst.org/twiki/bin/view/FST/WebHome [[Kategorie:Applications]] 6lsitev2q9gof8fupoldppu34pi05ko 0 280 4975 4900 2016-05-13T09:11:47Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Dendroscope|Česká verze--> ===Description=== '''Dendroscope''' is a user-friendly program for visualizing and navigating phylogenetic trees, for both small and large datasets. The software is written in Java 1.4. The program provides all standard tree visualizations and is optimized to run interactively on trees containing hundreds of thousands of taxa. The program provides tree editing and graphics export capabilities[1]. [1]Huson, D.H., Richter, D.C., Rausch, C., Dezulian, T., Franz, M., Rupp, R. (2007). Dendroscope: An interactive viewer for large phylogenetic trees. BMC Bioinformatics. 8, 460. ===Availability=== '''Module <code>dendroscope-3.2.8</code>:''' ''Dendroscope version 3.2.8'' Every user is required to ask for a [http://www-ab2.informatik.uni-tuebingen.de/software/dendroscope/register/ licence key via the producer's webpage]. ===Use=== ssh -X skirit.metacentrum.cz qsub -X -I -q short -l nodes=1:ppn=1 -l mem=2gb module add dendroscope-3.2.8 ; Dendroscope ===Documentation=== http://ab.inf.uni-tuebingen.de/software/dendroscope/ ===Licence=== free for academic use (requires a licence key -- see above) ===Supported platforms=== amd64 ===Program administrator=== Petr Navrátil (ÚEB AV ČR), Tom Rebok [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.uni-tuebingen.de/ [[Kategorie:Applications]] cua29qhbkcjgfpg7bqr36tzqj5s98ra 0 385 4977 4793 2016-05-13T09:11:50Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--CEGMA|Česká verze--> = Popis = CEGMA (Core Eukaryotic Genes Mapping Approach) is a pipeline for building a set of high reliable set of gene annotations in virtually any eukaryotic genome. The strategy relies on a simple fact: some highly conserved proteins are encoded in essentially all eukaryotic genomes. We use the KOGs database to build a set of these highly conserved ubiquitous proteins. We define a set of 458 core proteins, and the protocol, CEGMA, to find orthologs of the core proteins in new genomes and to determine their exon-intron structures. The procedure uses information from the core genes of six model organisms by first using TBLASTN to identify candidate regions in a new genome. It then proposes and redefines gene structures using a combination of GeneWise, HMMER and geneid. The system includes the use of a profile for each core protein to ensure the reliability of the gene structure. = Availability= Version 2.5 Application is available for all users of MetaCentrum. = Use= Load module: module add cegma-2.5 Afterwards you can submit following command cegma You will see list of parametrs which you can use to run the appliacation. If you use multiple cores specify the number of threads by option <code>--threads </code>. You can use variable $PBS_NUM_PPN to get the number of reserved CPU. --threads $PBS_NUM_PPN = Documentation = On-line documentation you can find on http://korflab.ucdavis.edu/Datasets/cegma Original article is on http://bioinformatics.oxfordjournals.org/content/23/9/1061.full.pdf+html = Licence= GNU GENERAL PUBLIC LICENSE = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://korflab.ucdavis.edu/Datasets/cegma [[Category:Applications]] [[Category:English]] daei9sb8bvwd4djdrv4rlmq308flss4 0 145 4978 4892 2016-05-13T09:11:51Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Autodock_Vina|Česká verze--> We thank Mgr. Jiří Wiesner for writting following text. ===Description=== AutoDock Vina is a new program for drug discovery, molecular docking and virtual screening, offering partial receptor flexibility, multi-core capability, high performance and enhanced accuracy and ease of use. AutoDock Vina achieves an approximately two orders of magnitude speed-up compared with the molecular docking software AutoDock 4, while also significantly improving the accuracy of the binding mode predictions. On a typical modern workstation, Vina should often be able to reliably generate the predicted binding modes and affinities of a ligand in seconds to minutes, depending on the ligand complexity. ===Reference=== O. Trott, A. J. Olson, AutoDock Vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization and multithreading, Journal of Computational Chemistry (in press) ===Availability=== Program is actually available in version 1.0.2 and 1.1.2 ===Use=== The current versions available in Meta centrum are vina-1.0.2 and vina-1.1.2. The executable is 32-bit, but runs on 64-bit systems with 32-bit compatibility libraries. ===Documentation=== *[http://vina.scripps.edu/tutorial.html AutoDock Vina - Tutorial] *[http://vina.scripps.edu/manual.html AutoDock Vina - Manual] ===Licence=== User have to agree with [http://metavo.metacentrum.cz/en/myaccount/licence.html licence] ===Supported platforms=== Linux ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://vina.scripps.edu/ ===Detailed documentation=== IMPORTANT!! 0. Do I always have to specify the number of CPUs being used by Vina? Yes, unless you are using the whole node for that particular job. Vina, if not restricted, automatically claims to use all processors on the node. ====Preparation==== 1. What input format does Vina use for receptor and ligand files? The input format is PDBQT format, which is also used by Autodock4. PDBQT format is an extended pdb format with atomic charge and atom type information located in separate columns behind the coordinate columns. 2. What is the preferred application for preparation of PDBQT files? AutoDockTools which is a part of MGLTools, web page: http://mgltools.scripps.edu. Search for the detailed instructions on the pdbqt conversion on this web page: http://autodock.scripps.edu/faqs-help/tutorial/using-autodock-4-with-autodocktools. The instructions on preparation of flexible receptor sidechains are also included. 3. What kind of charges should the PDBQT files contain? Gasteiger, RESP on HF/6-31G* level? The charges may be zero in fact, Vina does not use any charges at all. 4. How do I define the search space? Vina uses a grid, does it? Yes, Vina uses a grid which is defined by its center and number of points in each dimension. The grid spacing is always set to 1A and its purpose is just to inform Vina about the size of the search space; it has nothing to do with the accuracy of the computation. AutoDockTools is the preferred application to visualize the grid - search space, see the above mentioned tutorial. ====Usage==== 1. How do I run a calculation? The calculation can be set up by using command line arguments or a configuration file. To display the available command line arguments type "vina --help". The configuration file syntax is: "command_line_argument = value", each argument on its separate line. To run a calculation with a config file type "vina --config CONFIG_FILE_NAME". Configuration file, example: receptor = ../../1xlw_spo_rigid.pdbqt flex = ../../1xlw_spo_flex.pdbqt ligand = ../../K05.pdbqt center_x = -2.518 center_y = 4.689 center_z = -9.073 size_x = 28 size_y = 32 size_z = 28 out = K05_out.pdbqt cpu = 1 exhaustiveness = 8 2. What does the exhaustiveness parameter mean? It influences the thoroughness of the global search algorithm. Larger values increase the probability of finding the global minimum, but also extend the computational time. Increasing the exhaustiveness value increases the time linearly and decreases the probability of not finding the minimum exponentially. Apart from exhaustiveness influenced by users, Vina has an internal heuristic algorithm to extend the search in accordance with an increasing number of atoms and rotatable bonds. 3. How long does a typical Vina docking run take? Exhaustiveness 8, 20 rotatable torsions, need about 14 minutes on one processor. 4. But... There is something wrong! AFAIK, AutoDock4 used grid too, but the grid has to be calculated in advance, it also uses grid configuration file and docking configuration file... and things are much more complicated. How come? Vina uses grid to compute energies, but it automatically calculates the grid maps, no other configuration files, except the one described above and even that is optional, are need. Yes is really so simple! Vina also clusters the results for you, hence no endeavors in this direction are requested from you either. ====Visualization==== 1. How do display the docking results? Use Pymol. PDBQT files can be directly opened, you have to display them by displaying all files. Use arrows to switch between different ligand binding modes. 2. I see incorrectly oriented hydrogens in the ligand and flexible receptor parts. How is this possible? Vina uses a united-atom scoring function. In Vina, the degrees of freedom that only move hydrogens, such as the hydroxyl group torsions, are degenerate. Therefore, in the output, some hydrogen atoms can be expected to be positioned randomly (but consistent with the covalent structure). For a united-atom treatment, this is essentially a cosmetic issue. [[Kategorie:Applications]] 1h9xd02tczjz3l1nxweo5mj34ruqrdo 0 148 4983 4891 2016-05-13T09:11:54Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--DeMon|Česká verze--> ===Description=== deMon (density of Montréal) is a software package for density functional theory (DFT) calculations. It uses the linear combination of Gaussian-type orbital (LCGTO) approach for the self-consistent solution of the Kohn-Sham (KS) DFT equations. The calculation of the four-center electron repulsion integrals is avoided by introducing an auxiliary function basis for the variational fitting of the Coulomb potential. Some of the features of the deMon package are: *Variational fitting of the Coulomb potential *Geometry optimization and transition state search *Molecular dynamic simulations (MD) *Time-dependent DFT (TD-DFT) *Calculation of properties like polarizabilities, hyperpolarizabilities, NMR, IR and Raman spectra and intensities, thermodynamic data *Parallelized code (MPI) *Interfaces for visualization software (Molden, Molekel, Vu) *Portability to various computer platforms and operating systems ===Availability=== Program is actually available in version demon 2.3 and demon-2.3-shmem, -shmem means parallel version using MPI above shared memory (for single machines with more processors). ===Use=== Load module ("module ad demon-2.3" or "module ad demon-2.3-shmem") and run program. ===Documentation=== [http://www.demon-software.com/public_html/support.html#manual On-line] documentation is available. ===Supported platforms=== Linux ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://www.demon-software.com/public_html/index.html [[Kategorie:Applications]] 8keeem4k50twuwcg0ccspio7mgt4z05 0 374 4987 4727 2016-05-13T09:11:56Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Newick Utilities|Česká verze--> ===Description=== The Newick Utilities are a suite of Unix shell tools for processing phylogenetic trees. Functions include re-rooting, extracting subtrees, trimming, pruning, condensing, drawing (ASCII graphics or SVG).. ===Availability=== Program je dostupný ve verzi 1.6 ===Use=== module add newick-utils-1.6 aplikace: nw_clade nw_distance nw_gen nw_match nw_rename nw_support nw_condense nw_duration nw_indent nw_order nw_reroot nw_topology nw_display nw_ed nw_labels nw_prune nw_stats nw_trim ===Licence=== BSD ===Documentation=== Dokumentace je dostupná na http://cegg.unige.ch/newick_utils ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://cegg.unige.ch/newick_utils [[Kategorie:Applications]] kr385zmggi7b2kqxqhsx3f2moss17p9 0 184 4988 4905 2016-05-13T09:11:57Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Vývojové_prostředí_SGI|Česká verze--> ===Description=== SCollection of development tools of SGI company for IRIX contains compilers MIPSPro laguages C, C++, Fortran 77, Fortran 99 with paralleling possibilities (OpenMP, MPI, SHMEM), line debugger and profiler (dbx, prof) and inegred graphical enviroment ProDev WorkShop, which contains GUI debugger (cvd), tools for analyse of program structure (cvstatic, cvbuild), Test coverage analysis (cvcov) analyse of program power characteristic (cvperf, cvmeter), test coverage analysis (cvcov) and analyse of program power characteristic (cvperf, cvmeter). ===Use=== All tools are available directly in standart enviroment (are installed in system directories) according howto from documentation. ===Documentation=== Documentation is available on [http://techpubs.sgi.com/library/tpl/cgi-bin/browse.cgi?coll=0650&db=bks&pth=/SGI_Developer WWW server] of producer or locally in documentation system Infosearch/SGIHelp. ===Licence=== Every user can use this program in the MetaCentrum machines with SGI IRIX system. ===Supported platforms=== SGI IRIX ===Program administrator=== [mailto:salvet@ics.muni.cz Zdeněk Salvet] ===Homepage=== URL: http://www.sgi.com/developers/technology/irix/index.html [[Kategorie:Applications]] 8ar62o8skhxvf88590tabafomt014kk 0 290 4989 4735 2016-05-13T09:12:00Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--ECHO|Česká verze--> = Description = ECHO is an error correction algorithm designed for short-reads from next-generation sequencing platforms such as Illumina's Genome Analyzer II. The algorithm uses a Bayesian framework to improve the quality of the reads in a given data set by employing maximum a posteriori estimation. = Availability = Version 1.12 = Licence = BSD license = Use = Example of shell script: #!/bin/bash #PBS -N myecho #PBS -l nodes=1:ppn=8 #PBS -l mem=10g #PBS -j oe # let's initialize modules . /packages/run/modules-2.0/init/sh trap "clean_scratch" TERM EXIT module add echo-1.12 # let's set the necessary variables DATADIR="/storage/brno2/home/$LOGNAME/" # let's copy the input data cp -r $DATADIR/100000.fastq $SCRATCHDIR # let's change the working directory cd $SCRATCHDIR # let's perform the computation ECHO -k 15 -u $PBS_NUM_PPN -b 300000 --nh 256 -o 1k-fixed 100000.fastq # let's copy-out the result cp 1k-fixed $DATADIR || export CLEAN_SCRATCH=false = Documentation = http://uc-echo.sourceforge.net/README.txt = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://uc-echo.sourceforge.net/ [[Category:Applications]] [[Category:English]] br7hvpppv42twmfnegg9k8yrtvrk3af 0 321 4993 4815 2016-05-13T09:12:05Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--GNUplot|Česká verze--> =Description= '''Gnuplot''' is a command-line driven graphing utility supporting many types of plots in either 2D and 3D. It can draw using lines, points, boxes, contours, vector fields, surfaces, and various associated text. It also supports various specialized plot types as well as many different types of output: interactive screen terminals (with mouse and hotkey input), direct output to pen plotters or modern printers, and output to many file formats (eps, emf, fig, jpeg, LaTeX, pdf, png, postscript, ...). Gnuplot is easily extensible to include new output modes. =Availability= Currently, the version '''4.6.4''' is available. =Use= Load the module ("<code>module ad gnuplot-4.6.4</code>") and run the program <code>gnuplot</code>. If you need to work interactively and have a graphical program output, it is necessary to use the [[Remote_desktop|provided graphical environment]]. {{Application with GUI}} =Documentation= The [http://www.gnuplot.info/documentation.html on-line] dokumentation is available. =Program administrator= Tom Rebok ([mailto:meta@cesnet.cz meta@cesnet.cz]) =Homepage= URL: http://www.gnuplot.info/ [[Kategorie:Applications]] [[Kategorie:English]] 24pbvsyf1buugact19xxnuzzirs3hw7 0 262 4998 4818 2016-05-13T09:12:12Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--CP2K|Česká verze--> = Description = CP2K is a program to perform atomistic and molecular simulations of solid state, liquid, molecular, and biological systems. It provides a general framework for different methods such as e.g., density functional theory (DFT) using a mixed Gaussian and plane waves approach (GPW) and classical pair and many-body potentials. = Availability = Version 2.3 (without infiniband support) and 2.4 (with infiniband support). Freely available to users. Modules: cp2k -> cp2k-2.3 cp2k-2.4 = Licence = [http://www.gnu.org/copyleft/gpl.html GPLv3] = Use = Initialize environment with command: module add cp2k-2.4 Initialization makes available also MPI implementation and system variable $CP2K pointing into CP2K install dir. Usage of one of the tools with sample data: qsub -I -q short -l mem=4gb -l nodes=2:ppn=4:debian60 -l scratch=300mb:shared module add cp2k-2.4 cd $SCRATCHDIR cp -r $CP2K/tests . cd tests/TAMC/regtest mpirun -np 8 $CP2K_BIN ./dimer.inp {{MPI application}} = Documentation = Dokumentation is available at http://www.cp2k.org/index.php/documentation, and in directory $CP2K/doc. = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://www.cp2k.org/ [[Category:Applications]] [[Category:English]] ci07q23qq9bjdejwx8yfenyns0c5fd1 0 303 5002 4996 2016-05-13T09:12:15Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--rNA-mrNA|Česká verze--> =Description= rNA (randomized Numerical Aligner) is a software able to align the huge amount of data produced by Next Generation Sequencers. The main feature of rNA is the fact that it achieves an accuracy greater than the majority of other tools in a feasible amount of time. rNA works with single as well as paired ends reads and it allows indels and delta-search for better accuracy. A graphical user interface is provided for not command-line-skilled users and an MPI-based version for cluster is in development. '''Note:''' program has ceased development. There is a rewritten application instead called [[ERNE application|ERNE]] which is also available in MetaCentrum. =Availability= Program is actually available in version 1.0. Module versions: mrNA-1.0-gcc mrNA-1.0-intel Intel version should be faster, GCC version maybe more stable. =Licence= [http://www.gnu.org/licenses/gpl.html GNU/GPL v3 licence] =Use= Initialize environment by command module add mrNA-1.0-intel This makes available the path to program binaries. =Documentation= Documentation is available in file <tt>/software/mrNA-1.0/gcc/share/doc/rna/README</tt>. =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://iga-rna.sourceforge.net/ [[Category:Applications]] [[Category:English]] o7pksx63c0b955begbmczuxnllss3sj 0 286 5003 4844 2016-05-13T09:12:16Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--PyFITS|Česká verze--> ===Description=== '''PyFITS''' module is a Python library providing access to FITS files. FITS (Flexible Image Transport System) is a portable file standard widely used in the astronomy community to store images and tables. ===Availability=== * '''PyFITS version 3.1.2:''' uses Numpy compiled with Intel MKL 11.0 '''(internally-paralelized)''' ** module <code>pyfits-3.1.2-py2.7</code> *** ready for use with Python 2.7 ===Use=== Load the module (<code>module ad "modulename"</code>), then you may import the PyFITS libraries to Python (<code>import pyfits</code>). ===Documentation=== Just the [http://stsdas.stsci.edu/download/docs/The_PyFITS_Handbook.pdf on-line] documentation is available. ===Supported platforms=== Linux ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.stsci.edu/institute/software_hardware/pyfits [[Kategorie:Applications]] cgs63zbfs53v963u07zho0mo70upi8u 0 274 5005 4768 2016-05-13T09:12:19Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--SNAPP|Česká verze--> =Description= SNAPP (SNP and AFLP Package for Phylogenetic analysis) is package for inferring species trees and species demographics from independent (unlinked) biallelic markers such as well spaced SNPs. It implements a full coalescent model, but uses a novel algorithm to integrate over all possible gene trees, rather than sampling them explicitly. Like [[BEAST application|BEAST]], a SNAPP analysis is controlled using a specially formatted XML file. This file can be created from scratch, or by using the graphical user interface in BEAUTi. SNAPP can be run directly from inside BEAUTi, or you can take the xml file and use it to run SNAPP on any computer or server. Like BEAST and MrBayes, SNAPP does not output a single tree. Rather, it uses Markov chain Monte Carlo (MCMC) to generate multiple trees (and corresponding parameter values), each of which is a sample from the posterior distribution of species trees and parameters. The trees and parameter values are output as two files. As with all MCMC analyses, care must be taken when interpreting these outputs. For example, it is necessary to check convergence and be prepared to consider multiple plausible species trees, instead of just one. The file containing parameter values can be analysed using the Tracer software. As with all MCMC software, it is advisable to run multiple analyses, possibly tweaking proposal parameters, to validate an analysis. =Availability= Program is actually available in version 1.1.1 Module versions: snapp -> snapp-1.1.1 snapp-1.1.1 =Licence= [http://www.gnu.org/licenses/gpl.html GNU/GPL licence] =Use= Initialize environment by command module add snapp This makes available the system variable $SNAPP which points to home directory of application. Also it makes available the module <tt>[[JDK application|jdkibm-7]]</tt> so the JAVA platform can be used. Example of work: <source lang="bash"> $ snapp -help Usage: snapp [-window] [-options] [-working] [-seed] [-prefix <PREFIX>] [-overwrite] [-resume] [-errors <i>] [-threads <i>] [-help] [<input-file-name>] -window Provide a console window -options Display an options dialog -working Change working directory to input files directory -seed Specify a random number generator seed -prefix Specify a prefix for all output log filenames -overwrite Allow overwriting of log files -resume Allow appending of log files -errors Specify maximum number of numerical errors before stopping -threads The number of computational threads to use (default auto) -help Print this information and stop Example: snapp test.xml Example: snapp -window test.xml Example: snapp -help </source> {{Application with GUI}} =Documentation= Documentation is available in http://snap-mcmc.googlecode.com/files/SNAPPv1.1.pdf, or in file <tt>/software/snapp-1.1.1/doc/SNAPP.pdf</tt>. =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://beast2.cs.auckland.ac.nz/index.php/SNAPP [[Category:Applications]] [[Category:English]] 379e2d4wto3mckry9th15swhsscwlz0 0 294 5007 4901 2016-05-13T09:12:24Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--GenomeAnalysisTK (GATK)|Česká verze--> = Description = The Genome Analysis Toolkit or GATK is a software package developed at the Broad Institute to analyse next-generation resequencing data. The toolkit offers a wide variety of tools, with a primary focus on variant discovery and genotyping as well as strong emphasis on data quality assurance. Its robust architecture, powerful processing engine and high-performance computing features make it capable of taking on projects of any size. = Availability = Version 2.7-2. Freely available to users. Available modules: gatk-2.7.2 = Licence = [http://www.broadinstitute.org/gatk/about/#licensing Own licence] freely available for academic users. = Use = Example of environment initialization: module add gatk-2.7.2 Initialization makes available also java 7 and system variable $GATK pointing into GATK install dir. Usage of one of the tools with sample data: java -Xmx2g -jar $GATK/GenomeAnalysisTK.jar -T CountReads -R $GATK/resources/exampleFASTA.fasta -I $GATK/resources/exampleBAM.bam = Documentation = Dokumentation is available at http://www.broadinstitute.org/gatk/guide/ . = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://www.broadinstitute.org/gatk [[Category:Applications]] [[Category:English]] i8ddqe266kp7y74j7ggkhannxoxwewi 0 143 5008 4757 2016-05-13T09:12:26Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Amber|Česká verze--> ===Description=== Program Amber is a set of molecular mechanical force fields for the simulation of biomolecules (biologically important macromolecules – especially proteins and nucleic acids) The program package is created by package of molecular simulation programs, computings of molecular mechanics, simulation of molecular dynamics with tools for analyze and visualization results (program Moil-View). Program package contains cca 50 programs, which cover wide spectrum of computational chemistry. Main programs are following Amber modules: '''sander''':Simulated annealing with NMR-derived energy restraints. This allows for NMR refinement based on NOE-derived distance restraints, torsion angle restraints, and penalty functions based on chemical shifts and NOESY volumes. Sander is also the "main" program used for molecular dynamics simulations, and is also used for replica-exchange, thermodynamic integration, and potential of mean force (PMF) calculations. Sander also includes QM/MM capability (before was in module roar '''pmemd''' : This is an extensively-modified version (prepared by Bob Duke) of the sander program, optimized for periodic, PME simulations, and for GB simulations. It is faster, and scales better on parallel machines, than sander; hence it is generally the program of choice, unless you need options that it does not support. '''nmode''' : Normal mode analysis program using first and second derivative information, used to find search for local minima, perform vibrational analysis, and search for transition states. '''LEaP''': LEaP is an X-windows-based program that provides for basic model building and Amber coordinate and parameter/topology input file creation. It includes a molecular editor which allows for building residues and manipulating molecules. '''antechamber''': This program suite automates the process of developing force field descriptors for most organic molecules. It starts with structures (usually in PDB format), and generates files that can be read into LEaP for use in molecular modeling. The force field description that is generated is designed to be compatible with the usual Amber force fields for proteins and nucleic acids. '''ptraj''': TThis is used to analyze MD trajectories, computing a variety of things, like RMS deviation from a reference structure, hydrogen bonding analysis, time-correlation functions, diffusional behavior, and so on. '''mm_pbsa''' : This is a script to automate post-processing of MD trajectories, to analyze energetics using continuum solvent ideas. It can be used to break energies energies into "pieces" arising from different residues, and to estimate free energy differences between conformational basins. ===Availability=== The following versions are available: * '''14.0 (modules <code>amber-14</code> and <code>amber-14-gpu</code>)''' ** version available '''for all the MetaCentrum and CERIT-SC users''' ** variant '''<code>amber-14</code>''' is ''compiled by Intel compiler with the support of Intel MKL and distributed (MPI) computing (OpenMPI)'' ** variant '''<code>amber-14-gpu</code>''' is ''compiled by Intel compiler with the support of Intel MKL, GPU computing (CUDA 6.0), and distributed (MPI) computing (OpenMPI)'' * 8.0, 10 and 12 (in the appropriate modules --- amber8, amber10, amber-12) ** version 12 is available just for the Masaryk University users ** furthermore, special parallel versions are available based on the module suffix, e.g.: *** shmem: parallel version using MPI above shared memory (for single machines with more processors) (module amber8-mpich-shmem) ===Use=== Load the appropriate Amber package based on the desired version (e.g., <code>module add amber-14</code>) and run the desired program in accordance with its documentation. ===Documentation=== [http://ambermd.org/ on-line] documentation is available. ===Licence=== * Amber v. 14 is licensed '''for all the MetaCentrum and CERIT-SC infrastructure users''' * Amber v. 10 is licensed for all the UK a UJP students/employees. * Amber v. 8 is licensed for all the MU students/employees. ===Supported platforms=== SGI Irix, Linux, IBM AIX, HP Unix, Sun OS, Max OS, MS Windows (with use program Cygwin), MD-GRAPE ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://ambermd.org/, http://www.emporia.edu/ ===Example=== <source lang="bash"> $ qsub -l nodes=2:ppn=4:x86_64,mem=1gb -q normal job.sh </source> where job.sh follows (lojza is login) <source lang="bash"> #!/bin/sh # # name of job #PBS -N job # # messaging of end of job #PBS -m ae # # one file for output and error (from PBS we get file job.o<number of job>) #PBS -j oe # # copy the Amber's input data from a shared NFSv4 storage to a scratch volume cp /storage/brno2/home/$USER/job.in $SCRATCHDIR # # change the working directory cd $SCRATCHDIR || exit 1 # # adding of module module add amber14 # # run of computation mpirun sander.MPI -O -i ./job.in -c molekula -o ./job.output # # copy out the Amber's output data to a shared NFSv4 storage cp job.output /storage/brno2/home/$USER/ # # clean the temporary storage rm -rf $SCRATCHDIR </source> [[Kategorie:Applications]] 3y3rcqc3zoshwrqi5z1n01aa0fdjetg 0 341 5010 4777 2016-05-13T09:12:28Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Mugsy|Česká verze--> = Description = Mugsy is a multiple whole genome aligner. Mugsy uses Nucmer for pairwise alignment, a custom graph based segmentation procedure for identifying collinear regions, and the segment-based progressive multiple alignment strategy from Seqan::TCoffee. Mugsy accepts draft genomes in the form of multi-FASTA files and does not require a reference genome. = Availability = Version v1r2.3. Freely available for every user. = Licence = [http://www.bioperl.org/wiki/Perl_Artistic_License Perl Artistic License] = Use = Initialize module with command: module add mugsy-v1r2.3 ====Running example==== mugsy --directory $SCRATCHDIR --prefix mygenomes xxx1.fasta xxx2.fasta xxx3.fasta = Documentation = Documentation is available at http://mugsy.sourceforge.net/#started. = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://mugsy.sourceforge.net/ [[Category:Applications]] [[Category:English]] 57vxclo515euz89vyyvlexceqhq0b0b 0 317 5011 4734 2016-05-13T09:12:30Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--LAMMPS|Česká verze--> = Description = Classical molecular dynamics code, and an acronym for Large-scale Atomic/Molecular Massively Parallel Simulator. LAMMPS has potentials for soft materials (biomolecules, polymers) and solid-state materials (metals, semiconductors) and coarse-grained or mesoscopic systems. It can be used to model atoms or, more generically, as a parallel particle simulator at the atomic, meso, or continuum scale. = Availability = Version from December 2013. Freely available to users. Modules: lammps-nov2015 lammps-dec2013 = Licence = [http://www.gnu.org/copyleft/gpl.html GPLv3] = Use = Initialize environment with command: module add lammps-dec2013 '''Note:''' There is a significant difference in computing speed of the program between Intel and AMD machine architecture. It runs better on Intel. [http://metavo.metacentrum.cz/pbsmon2/hardware >> list of MetaVO machines] {{MPI application}} ==== what includes lammps binary ==== The lammps binary includes many compiled parts. You can get their list by issuing command lammps -help If you need more components compiled inside, give us feedback. = Documentation = Documentation is available at http://lammps.sandia.gov/doc/Manual.html, tutorials are [http://lammps.sandia.gov/tutorials.html here], description of commands is [http://lammps.sandia.gov/doc/Section_commands.html#cmd_5 here]. = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://lammps.sandia.gov/ [[Category:Applications]] [[Category:English]] 04mjekgr93c4y3cv1ioe82g4vjfs9lv 0 235 5014 4966 2016-05-13T09:12:33Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Cap3-pCap|Česká verze--> ===Description=== Sequence Assembly Program ===Availability=== Without version. ===Use=== module add cap3-pcap cap3 pcap or other binaries (see ls /software/cap3-pcap/bin) ===Documentation=== http://seq.cs.iastate.edu/ ===Licence=== Freely available for use at non-profit organizations. ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://seq.cs.iastate.edu/ [[Kategorie:Applications]] glgh6shs8wemugjn8vkhkx7x7osvrnw 0 158 5017 4764 2016-05-13T09:12:36Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--VASP|Česká verze--> ===Description=== VAMP/VASP is a package for performing ab-initio quantum-mechanical molecular dynamics (MD) using pseudopotentials and a plane wave basis set. These techniques avoid all problems occurring in the original Car-Parrinello method. ===Availability=== Program is actually available in version 4.6 and 5.2. MetaCentrum does not own a licence for all users. According to VASP licence agreement we can make VASP available to every research group or institution, which proves the licence ownership. Please send us a scanned copy and we make the necessary steps to make the application available to you in the corresponding version. ===Use=== Load module ("module add X"), where X could be: vasp46, vasp46-parallel, vasp52, vasp52-parallel to run paralell program: <source lang="bash"> #environment initialisation lamboot cd ... mpirun C vasp ... #also more runs ("C" means run in all allocated processors) # environment cleaning lamhalt </source> ===Documentation=== [http://cms.mpi.univie.ac.at/vasp/vasp/vasp.html On-line] documentation is available. ===Licence=== Everybody interested in using this program has to send his licence to the email: [mailto:meta@cesnet.cz meta@cesnet.cz], then we check authenticity of licence and enable program to the user. ===Supported platforms=== Linux 64bit machines ( x86_64 property) ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://cms.mpi.univie.ac.at/vasp/ [[Kategorie:Applications]] tly05z8djqd3zme79w87y0joisc1hjd 0 181 5018 4930 2016-05-13T09:12:37Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--JDK|Česká verze--> ===Description=== JDK is a collection of Java development tools. Its part is also JRE, compiler and many other tools. Program extension Java 3D is also available, Java 3D enables possibility of using accelerated 3D graphics. There is also available JDK of companies SUN and IBM in versions 6, 7 and 8. Initialize modules in your batch script: <source lang="bash"> #!/bin/bash module add jdk-7 export CLASSPATH=/home/something/something.jar:/software/java-libs/javamail-1.4/mail.jar:/software/java-libs/jaf-1.1/activation.jar java cz.something.SomeClass </source> Available versions JDK differ according operating system of given machine (platform you get by command fs sys), there are following modules available for platform amd64_linux26: *java3d *jdk-1.6.0 *jdkibm-1.6.0 *jdk-7 *jdkibm-7 *jdk-8 For platform i386_linux26 is available jdk-1.6.0. ===Documentation=== Documentation is available on [http://java.sun.com/javase/downloads/index.jsp#docs server of producer]. ===Supported platforms=== Linux ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://java.sun.com/javase/ [[Kategorie:Applications]] o6zrhjbyeh98okc64f6goq9rgwdhj4g 0 223 5019 4772 2016-05-13T09:12:38Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--DMU|Česká verze--> ===Description=== '''''DMU''''' is a package of tools for analysing multivariate mixed models. The package consists of the following modules: * '''DMU1''' -- Initial program that always must be used; the program reads a driver file containing descriptions of data, model and variance structure, prior variances and covariances and auxiliary input to the other modules. * '''DMU4''' -- This module can be used to predict future outcomes of random effects (e.g. breeding values) and to estimate fixed effects. * '''DMU5''' -- This module can be used to solve the multiple trait mixed model equations based on iteration on data. * '''DMUAI''' -- This module can be used for estimation of (co)variance components using ''Average Information REstricted Maximum Likelihood (AI-REML)'' (Jensen et al. 1997). * '''RJMC''' -- Description to come (the module is still under development). ===Availability=== DMU is available in the version 6, release 5.0. ===Use=== '''Batch use:''' * prepare the job description script -- use a general skeleton supplemented by the following lines: <source lang="bash"> ... # load the module into the environment module add dmu-6.5 # run the computation # general format: PROGRAM <INPUT_FILE.DIR >OUTPUT_FILE # available programs: dmu1, dmu4, dmu5, dmuai, rjmc # e.g.: dmu1 <my_input_file.DIR >my_output_file ... </source> * pass the job description file to the scheduler together with the request for a single processor and the requested amount of memory: <source lang="bash"> $ qsub -l nodes=1:ppn=1 -l mem=XXXg mydescriptionfile.sh </source> '''Interactive use:''' * ask the scheduler for an interactive job having a single node and a single processor reserved: <source lang="bash"> $ qsub -I -l nodes=1:ppn=1 -l mem=XXXg </source> :''Note: Do not forget to apppropriately set the amount of requested memory (mem attribute) and/or another job requirements.'' * load the DMU module into the environment: <source lang="bash"> $ module add dmu-6.5 </source> * to run the computations provided by the module, use the wrapping scripts (named <code>r_dmu4</code>, <code>r_dmu5</code>, <code>r_dmuai</code> and <code>r_rjmc</code>) followed by a file specifying the computation to be done (so-called ''driver file''): <source lang="bash"> $ r_dmu4 my_computation # the DIR suffix will be added automatically $ r_dmu5 my_computation # the DIR suffix will be added automatically $ r_dmuai my_computation # the DIR suffix will be added automatically $ r_rjmc my_computation # the DIR suffix will be added automatically </source> :''Note: Even though the wrapping scripts exist, one can also run the provided tools (dmu1, dmu4, dmu5, dmuai, rjmc) directly.'' ===Documentation=== The documentation is available on the [http://www.dmu.agrsci.dk/dmuv6_guide.5.0.pdf producer's WWW server] or locally in the <code>/software/dmu-6.5/doc/</code> directory. ===License=== The program is free of charge for research purposes, but the use should be '''acknowledged in publications''' by reference to its manual. ===Supported platforms=== Linux ===Program administrator=== Ludmila Zavadilová, Tomáš Rebok [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://www.dmu.agrsci.dk/ [[Kategorie:Applications]] j88s9h5rtwgl4bj831hijinv4eo5e8d 0 305 5027 4786 2016-05-13T09:12:44Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--ANSYS|Česká verze--> = Description = '''ANSYS''' is a general nonlinear multiphysics software offering structural and thermodynamic analysis, continuum flow analysis, analysis of electrostatic and electromagnetic fields and acoustic analysis. All of these analysis may be performed on its own, or thanks to ANSYS' multiphysics approach all in one comprehensive analysis. ANSYS allows to perform not only validating calculations, but thanks to its parametric computation models even for sensitivity and optimization analysis and reliability analysis. ===== <u>Available products</u> ===== ''The following Ansys products'' are available within the MetaCentrum infrastructure: * '''Ansys CFD''' -- product combining two primary general-purpose fluids simulation products offered by ANSYS – '''Ansys Fluent''' and '''Ansys CFX'''. :* '''Ansys Fluent''' contains the broad physical modeling capabilities needed to model flow, turbulence, heat transfer, and reactions for industrial applications ranging from air flow over an aircraft wing to combustion in a furnace, from bubble columns to oil platforms, from blood flow to semiconductor manufacturing, and from clean room design to wastewater treatment plants. :* '''Ansys CFX''' is a general purpose fluid dynamics program for simulating all types of models concerning fluid flow, including effects of temperature(conductance, radiance). * '''Ansys Mechanical''' -- product for simulating of structural and thermodynamic tasks. The engineering simulation product provides a complete set of elements behavior, material models and equation solvers for a wide range of mechanical design problems. In addition, it offers thermal analysis and coupled-physics capabilities involving acoustic, piezoelectric, thermal–structural and thermo-electric analysis. All ANSYS tools can utilize at most 4 local processors for task computation; for more demanding tasks in multiprocess/multinode environment (e.g. grids) it's necessary to use additional tool – '''Ansys HPC''' – every of its licenses allows for distribution of computation to another available (local or remote) processor. = Availability = '''''Ansys version 16.2 (<code>module add ansys-16.2</code>)''''' * product is licensed '''for all registered users of national grid infrastructure of MetaCentrum''' :* '''without location restriction''' (usable in the whole area of the Czech Republic) ''Ansys version 16.1 (<code>module add ansys-16.1</code>)'' * product is licensed '''for all registered users of national grid infrastructure of MetaCentrum''' :* '''without location restriction''' (usable in the whole area of the Czech Republic) ''Ansys version 15.0 (<code>module add ansys-15</code>)'' * product is licensed '''for all registered users of national grid infrastructure of MetaCentrum''' :* '''without location restriction''' (usable in the whole area of the Czech Republic) ''Ansys version 14.5 (<code>module add ansys-14</code>)'' * product is licensed '''for all registered users of national grid infrastructure of MetaCentrum''' :* '''without location restriction''' (usable in the whole area of the Czech Republic) <u>''Number of available licenses (applies to all above mentioned versions):''</u> * '''Ansys Academic Research CFD''' (Ansys Fluent + Ansys CFX) in quantity of '''25 parallel runs''' * '''Ansys Academic Research Mechanical in quantity of '''5 parallel runs''' * additional tool '''Ansys HPC''' '''in quantity of 60 pieces (=> cores)''' * '''Available tools''' (complete list of available products is '''[[media:Ansys-available_products.pdf|HERE]]'''): ** ''FLUENT'' : command <code>fluent</code> ** ''CFX'' : command <code>cfx*</code> (e.g. <code>cfx5</code>, <code>cfx5launch</code>, <code>cfx5post</code>, etc.) ** ''CFD Post'' : command <code>cfdpost</code> ** ''ICEM-CFD'' : command <code>icemcfd</code> ** ''Workbench'' : command <code>runwb2</code> :'''''Note:''' There are 25 more available floating licenses for students and employees of ZČU in Pilsen ofAnsys version 7.1, which are accompanied by 3 licences of LS-DYNA software -- see more at [[Ansys_(JČU)|this site]]. {{Warning | All licenses are restricted to '''academical use only'''}} = Usage = Both tools will be discussed separately due to slight differences in their initialization. {{Application with GUI}} == '''Ansys Fluent''' == Ansys Fluent supports both interactive and batch use: ==== Interactive usage ==== * login to a frontend node of your choice ** with '''tunneled graphic display''' for intended use of grpahic environment: <code>$ ssh -X skirit.ics.muni.cz</code> ** standard way for text-only regime is: <code>$ ssh skirit.ics.muni.cz</code> * ask scheduling system for '''interactive task''' having allocated intended quantity of nodes and cores (in case of graphic regime add a parameter <code>-X</code>) <source lang="bash"> skirit$ qsub -I -l nodes=X:ppn=Y -l mem=Zgb # text-only regime skirit$ qsub -I -X -l nodes=X:ppn=Y -l mem=Zgb # grafic regime </source> * start Fluent and proceed according to onscreen instructions ** '''grafic regime:''' <code>$ fluent</code> ** '''text-only regime:''' <code>$ fluent -g</code> *** ''Note 1: You can choose either serial or parallel processing during initialization.'' ==== Batch usage ==== * '''login to''' a frontend node of your choice <source lang="bash"> $ ssh skirit.ics.muni.cz </source> * '''prepare input data''' for computation '''and task description script''' -- use its sample form and add following commands: ** '''''serial computation:''''' <source lang="bash"> module add ansys-16.2 fluent <version> -g -i input_file # serial initialization of Fluent </source> ::* ''Note: you can get list of available versions by running <code>$ fluent -v</code>'' :* '''''paralel/distributed computation:''''' <source lang="bash"> module add ansys-16.2 cpus=`cat $PBS_NODEFILE | wc -l` fluent <version> -t${cpus} -p -cnf=$PBS_NODEFILE -g < flow.input # paralel/distributed initialization of Fluentu </source> ::* ''Note: you can get list of available versions by running <code>$ fluent -v</code>'' * pass this created task description script to scheduling system along with number of demanded nodes/cores and amount of memory via: <source lang="bash"> skirit$ qsub -l nodes=X:ppn=Y -l mem=Zgb mytaskdescriptionscript.sh </source> == '''Ansys CFX''' == ==== Interactive usage ==== * connect with '''tunneled graphical display''' to a frontend node of your choice ** <code>$ ssh -X skirit.ics.muni.cz</code> * ask scheduling system for '''interactive task''' having allocated intended quantity of nodes and cores (add a parameter <code>-X</code> for graphical display tunneling) <source lang="bash"> skirit$ qsub -I -X -l nodes=X:ppn=Y -l mem=Zgb </source> * start CFX and proceed according to onscreen instructions ** <code>$ cfx5</code> ** resp. <code>$ cfx5launch</code> ==== Batch usage ==== * '''login to''' a frontend node of your choice <source lang="bash"> $ ssh skirit.ics.muni.cz </source> * '''prepare input data''' for computation '''and task description script''' -- use its sample form and add following commands: ** '''''serial computation:''''' <source lang="bash"> module add ansys-16.2 cfx5pre -batch inputfile.pre [...] cfx5solve -def inputfile [...] cfdpost -batch inputfile.cse [...] </source> :* '''''parallel/distributed computation:''''' :** please '''contact us'''(email <code>rebok (at) ics.muni.cz</code>) if you need '''parallel/distributed computation''' with Ansys CFX. :** This code can be useful for parallel computation <source lang="bash"> #### Create host list hl=`sort $PBS_NODEFILE | uniq -c | awk '{print $2"*"$1}' | tr '\n' ',' | sed 's/,$//'` #### Run the computation, e.g. cfx5solve -def 0.def -ccl 0.ccl -part-large -start-method "Platform MPI Distributed Parallel" -par-dist $hl </source> * pass this created task description script to scheduling system along with number of demanded nodes/cores and amount of memory via: <source lang="bash"> skirit$ qsub -l nodes=X:ppn=Y -l mem=Zgb mytaskdescriptionscript.sh </source> == '''Ansys Mechanical''' == = Licence requirements = for academic use only = Documentation = * software documentation is available '''locally in program's directory:''' ** ''version Ansys 16.2:'' <code>/software/ansys-16.2/doc/readme.html</code> ** ''version Ansys 16.1:'' <code>/software/ansys-16.1/doc/readme.html</code> ** ''version Ansys 15:'' <code>/software/ansys-15/doc/readme.html</code> ** ''version Ansys 14:'' <code>/software/ansys-14/doc/readme.html</code> * [http://www1.ansys.com/customer online documentation on producer's website] is available ''just for registered users'' = Software administrator = Tomáš Rebok (meta (at) cesnet.cz) [[Category:Applications]] [[Category:English]] tht8l75hh410iit9go844j6dmvta4st 0 386 5028 4843 2016-05-13T09:12:45Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Vzor stránky o aplikaci|Česká verze--> =Description= Short description of purpose and functionality of the application. =Availability= Availability of the application in modules. Notes to availability to users, use or nodes. =Use= Steps to run the application =Documentation= =Licence= =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= URL: http://www.r-project.org/ =Tips and useful information= =Useful links= [[Category:Applications]] [[Category:English]] qelvyjyxn6fly48od98xrtfxifcxl5c 0 227 5029 4947 2016-05-13T09:12:46Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--ViennaRNA|Česká verze--> ===Description=== The Vienna RNA Package consists of a C code library and several stand-alone programs for the prediction and comparison of RNA secondary structures. With the new release of version 2.0, we introduce the most recent nearest neighbor energy model for all free energy calculations. Additionally, most of the stand-alone programs included are now able to read FASTA formatted input data. This makes conversion of you data files, as necessary in previous releases, obsolete! ===Availability=== Version 2.0.7 ===Use=== module add viennaRNA-2.0.7 RNAfold or other scripts (see Documentation). ===Documentation=== http://www.tbi.univie.ac.at/~ronny/RNA/vrna2_manpages.html ===Licence=== Non specified. ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.tbi.univie.ac.at/~ronny/RNA/vrna2.html [[Kategorie:Applications]] j4mps9i5j3cxy97ix9z2qgcp6e77h8q 0 326 5030 4880 2016-05-13T09:12:47Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--OMNeT++|Česká verze--> = Description = OMNeT++ is an extensible, modular, component-based C++ simulation library and framework, primarily for building network simulators. "Network" is meant in a broader sense that includes wired and wireless communication networks, on-chip networks, queueing networks, and so on. Domain-specific functionality such as support for sensor networks, wireless ad-hoc networks, Internet protocols, performance modeling, photonic networks, etc., is provided by model frameworks, developed as independent projects. OMNeT++ offers an Eclipse-based IDE, a graphical runtime environment, and a host of other tools. There are extensions for real-time simulation, network emulation, alternative programming languages (Java, C#), database integration, SystemC integration, and several other functions. = Availability = Version 4.4. Freely available to all users. Modules: omnetpp-4.4 = Licence = [http://www.omnetpp.org/home/license Own academic licence] = Use = Initialize environment with command: module add omnetpp-4.4 {{MPI application}} {{Application with GUI}} = Documentation = Documentation is at http://www.omnetpp.org/documentation. = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://www.omnetpp.org/ [[Category:Applications]] [[Category:English]] kpa3bus1eqnca502tqql67u4fbmz663 0 377 5034 4832 2016-05-13T09:12:52Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--jvarkit|Česká verze--> ===Description=== Java utilities for Bioinformatics ===Availability=== jvarkit-1.128 ===Use=== module add jvarkit-1.128 bamstats04 It is possible to compile more tools. For usage please check documentation. ===Documentation=== https://github.com/lindenb/jvarkit ===Licence=== The MIT License ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== https://github.com/lindenb/jvarkit [[Kategorie:Applications]] kf869l9uj8qifnkn82gl1fzxqx40qo9 0 153 5036 4767 2016-05-13T09:12:55Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--MolPro|Česká verze--> ===Description=== '''Molpro''' is a complete system of ab initio programs for molecular electronic structure calculations. As distinct from other commonly used quantum chemistry packages, the emphasis is on highly accurate computations, with extensive treatment of the electron correlation problem through the multiconfiguration-reference CI, coupled cluster and associated methods. ===Availability=== * '''module <code>molpro-2012.1</code>''' -- MolPro '''version 2012.1'''; compiled with OpenMPI, OpenMP and Intel MKL support. * module <code>molpro-2008.1</code> -- MolPro version 2008.1; o utilize this program, you have to buy a licence, then scan this licence and send it to address [mailto:meta@cesnet.cz meta@cesnet.cz] * module '''<code>molpro</code>''' refers to '''the newest available version''' ===Use=== Load the module and run the computation: <source lang=bash> $ module add molpro $ molpro inputfile.com </source> * the program automatically detects the number of available processors/cores (for both parallel as well as distributed computations) ** if necessary, the program automatically uses the MPI routines in order to perform distributed computation * to limit the number of processes/threads, use the <code>-n</code> and/or <code>-t</code> options -- see <code>molpro -h</code> or program documentation ===Documentation=== Available [http://www.molpro.net/info/current/doc/manual/?portal=user&choice=User%27s+manual on-line on producer's webpage] or locally in the program directory (<code>/software/molpro-2012.1/molprop_2012_1_Linux_x86_64_i8/doc/</code>). ===License=== ''The license of the '''MolPro version 2012.1''' is:'' * '''bought by CERIT-SC Centre''' for academical use by any MetaCentrum user * restricted just to Brno locality * requires users to properly reference the used modules (see the program documentation) ===Supported platforms=== Linux ===Program administrator=== Tom Rebok [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://www.molpro.net [[Kategorie:Applications]] jxs7e5xg96oau3elxk62h5jmoz0k01e 0 165 5224 5040 2016-05-13T09:20:22Z Root 1 wikitext text/x-wiki <!--([[MrBayes|Česká verze]])--> ===Description=== MrBayes is a program for the Bayesian estimation of phylogeny. Bayesian inference of phylogeny is based upon a quantity called the posterior probability distribution of trees, which is the probability of a tree conditioned on the observations. The conditioning is accomplished using Bayes's theorem. The posterior probability distribution of trees is impossible to calculate analytically; instead, MrBayes uses a simulation technique called Markov Chain Monte Carlo (or MCMC) to approximate the posterior probabilities of trees. Program runs in the command line. There are four steps to a typical Bayesian phylogenetic analysis using MrBayes: *Read the Nexus data file *Set the evolutionary model *Run the analysis *Summarize the samples ===Availability=== Version 3.1, beta 3.2, 3.2, 3.2.1, 3.2.2 (single processor version and also MPI version – 3.1 compiled with mpich-p4 and 3.2 with openmpi, you have to run it through mpiexec with loaded appropriate MPI modul), 3.2.3, 3.2.4 and '''3.2.6'''. Dirichlet priors version 3.1.2 and GPU version is compiled with openmpi. ===Use=== Load module ("module ad mrbayes") and run "mb" for one processor version or "mpiexec mb-mpi" for mpi version with more processors. Modules available: *mrbayes *mrbayes-pre3.2 (beta version) *mrbayes-3.2 *mrbayes-3.2.1 *mrbayes-3.2.2 *mrbayes-3.2.3 (compiled with intel compilers) *mrbayes-3.2.4 (compiled with intel compilers) *'''mrbayes-3.2.6''' (compiled with intel compilers) *mrbayes-3.1.2dir *mrbayes-gpu-2.1.1 (guide for [[GPU_clusters]]) E.g. <source lang="bash"> qsub -I -l nodes=1:ppn=2 -l mem=1gb module add mrbayes-3.2.4 mpirun mb-mpi myInput.nex </source> ===Documentation=== http://mrbayes.csit.fsu.edu/wiki/index.php/Manual ===Supported platforms=== amd64_linux26 ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://mrbayes.csit.fsu.edu/index.php ===Problems=== Parallel versions 3.2 and 3.2.1 do not product correct trees, see http://sourceforge.net/projects/mrbayes/develop When you are running version 3.1 and if you see error messages at the end of your output file like this <source lang="bash"> p0_7950: p4_error: interrupt SIGSEGV: 11 p3_7957: p4_error: net_recv read: probable EOF on socket: 1 rm_l_3_7977: (41.703125) net_send: could not write to fd=5, errno = 32 p4_7967: p4_error: net_recv read: probable EOF on socket: 1 rm_l_4_7978: (41.699219) net_send: could not write to fd=5, errno = 32 p5_7969: p4_error: net_recv read: probable EOF on socket: 1 rm_l_5_7979: (41.699219) net_send: could not write to fd=5, errno = 32 p6_7971: p4_error: net_recv read: probable EOF on socket: 1 rm_l_6_7980: (41.699219) net_send: could not write to fd=5, errno = 32 rm_l_2_7976: p4_error: interrupt SIGx: 15 rm_l_2_7976: (41.722656) net_send: could not write to fd=7, errno = 32 p0_7950: (57.542969) net_send: could not write to fd=4, errno = 32 p4_7967: (55.699219) net_send: could not write to fd=5, errno = 32 p5_7969: (55.699219) net_send: could not write to fd=5, errno = 32 p3_7957: (55.707031) net_send: could not write to fd=5, errno = 32 p6_7971: (55.707031) net_send: could not write to fd=5, errno = 32 mpiexec: Warning: tasks 0,3-6 exited with status 1. </source> use version 3.2. ===Example=== The beginner guide is available http://cluster.prf.jcu.cz/index.php/guides/general/bayesfromscratch The common begin of nex file for analysis is <source lang="bash"> #NEXUS [saved by seaview on Wed Oct 26 08:34:48 2011] BEGIN DATA; DIMENSIONS NTAX=52 NCHAR=1153; FORMAT DATATYPE=DNA GAP=- ; MATRIX [1] TCCMP1185 cgaaagcctgacggagca... </source> and the end of file <source lang="bash"> ...cctcctt ; begin mrbayes; [ Set the parameters of the likelihood model, keeping prset at default conditions ] lset nst=6 nucmodel=4by4 code=universal rates=gamma ngammacat=4; [ Set the outgroup for the analysis ] outgroup 52; [ Set Markov chain Monte Carlo parameters ] mcmcp ngen=50000 nruns=2 swapfreq=5 printfreq=100 samplefreq=100 nchains=4 savebrlens=yes ordertaxa=no filename=vysl-BT8; [ Go! ] mcmc; sump burnin=250; sumt burnin=250; end; </source> Lines at the and (sump and sumt) are mostly necessary for 3.2 version, because they summarize the results. ===Use with GNU Parallel=== If you need to run more MrBayes runs in parallel (for example for more genes separately), one of possibilities is use of [[GNU Parallel application]], which is library allowing user to launch more parallel processes on more CPU cores or computers. Basic use can look like this: <source lang="bash"> module add parallel module add mrbayes-3.2.2 ls *.nexus | parallel -j 10 'echo Start > {}.log && date >> {}.log && mb {} | tee -a {}.log && echo End: >> {}.log && date >> {}.log' </source> Parameter "-j 10" says to use 10 CPU cores. See manual of the function. Remaining commands produce nice logs with time stamps. The easiest example: <source lang="bash"> ls *.nexus | parallel -j 10 'mb {}' </source> [[Kategorie:Applications]] lz7ik6r7332tone5pv9b3wt21bwwd4b 0 379 5041 5004 2016-05-13T09:13:00Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--STAR|Česká verze--> =Description= Application STAR (Spliced Transcripts Alignment to a Reference) can by used for the detection and characterization of sliced RNAs. =Availability= Version: * 2.5.0c (modul <code>star-2.5.0c</code>) * 2.4.1b (modul <code>star-2.4.1b</code>) =Use= Example of module initialization and run: <source lang="bash"> $ module add star-2.4.1b $ STAR --option1 value1 --option2 value2 </source> The application supports parallel computing. For this reason the important option is <code>--runThreadN</code>. This option the defines the number of threads to be used for genome generation. By default it is 1. Other parameters are described in the [http://github.com/alexdobin/STAR/raw/master/doc/STARmanual.pdf STAR manual]. It is appropriate to set the number of threads to value of the variable <code>PBS_NUM_PPN</code>: <source lang="bash">STAR --runThread $PBS_NUM_PPN ...</source> =Documentation= * Online documentation [http://github.com/alexdobin/STAR/raw/master/doc/STARmanual.pdf STAR manual]. * Paper about the application and used algorithms: [http://bioinformatics.oxfordjournals.org/content/29/1/15 STAR: ultrafast universal RNA-seq aligner] * [http://groups.google.com/forum/#!forum/rna-star Forum/author's support] =Licence= STAR is a free open source software distributed under GPLv3. =Program manager= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= [https://github.com/alexdobin/STAR https://github.com/alexdobin/STAR] [[Category:Applications]] [[Category:English]] piusc3fe6aummjt202j1kql276pizm0 0 313 5043 4752 2016-05-13T09:13:01Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--SAS|Česká verze--> = Description = '''Science Analysis System (SAS)''' is ... = Availability = = Licence = = Use = = Documentation = = Program manager = Filip Münz, Matúš Kocka, Tomáš Rebok ([mailto:meta@cesnet.cz meta@cesnet.cz]) = Homepage = http://xmm.esac.esa.int/sas/current/documentation/sas_concise.shtml [[Category:Applications]] [[Category:English]] 7v0qm86uxhtpctjmps9kwce8kscaes0 0 373 5042 4893 2016-05-13T09:13:00Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--RevBayes|Česká verze--> = Description = Bayesian phylogenetic inference using probabilistic graphical models and an interpreted language. = Availability = Verze 1.0.0 beta (cca z 1.2.2015). Moduly: revbayes-022015 = Licence = [http://www.gnu.org/copyleft/gpl.html GPL v3] = Use = Add the module: module add revbayes-022015 Then you can run the RevBayes environment by issuing command: rb-mpi or mpirun /software/revbayes-022015/bin/rb-mpi {{MPI application}} ===Environment configuration=== You can set your configuration by RevBayes inner commands. This configuration is automatically written in file <tt>~/.RevBayes.ini</tt>. Initial RB environment configuration can be set by command: rb-mpi /software/revbayes-022015/src_orig/env_setup.nex = Documentation = * https://cdn.rawgit.com/revbayes/revbayes/master/help_html/index.html &ndash; command docs * https://github.com/revbayes/revbayes_tutorial/tree/master/tutorial_TeX &ndash; various PDF docs for environment usage = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://revbayes.github.io/about.html [[Category:Applications]] [[Category:English]] kgwzabga71qmoaw95npnhjttg3wylba 0 277 5044 4873 2016-05-13T09:13:02Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--GenomeTools|Česká verze--> =Description= The GenomeTools genome analysis system is a free collection of bioinformatics tools (in the realm of genome informatics) combined into a single binary named gt. It is based on a C library named “libgenometools” which consists of several modules. Includes also the tool LTRdigest. =Availability= Version 1.6.1. Freely available to all users. The modules have names genometools-1.3.3 (ltrdigest-1.3.3) genometools-1.5.1-gcc (ltrdigest-1.5.1-gcc) =Licence= [http://genometools.org/license.html Free license] based on OpenBSD model. =Use= You can load the module via the command module add genometools-1.3.3 The module load makes available the program path and also paths to openmpi-1.6, which enables job parallelization. =Documentation= http://genometools.org/documentation.html and http://www.zbh.uni-hamburg.de/?id=207. Except that is available the [http://www.zbh.uni-hamburg.de/fileadmin/gi/LTRdigest/ltrdigestman.pdf LTRdigest manual]. When not on web anymore, use the local copy in directory <tt>/software/ltrdigest-1.5.1/docs</tt>. ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://genometools.org and http://www.zbh.uni-hamburg.de/?id=207 [[Category:Applications]] [[Category:English]] fd6tlnh9p15o5bkja6gfvlwhcumd2d5 0 222 5045 4895 2016-05-13T09:13:03Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--BLUPF90|Česká verze--> ===Description=== '''''BLUPF90''''' family of programs is a collection of software in Fortran 90/95 for mixed model computations in animal breeding. For general description, see the paper ''I. Misztal: Complex Models, More Data: Simpler Programming?'' (available [http://www-interbull.slu.se/bulletins/bulletin20/BUL20_5.PDF here]) or the paper ''I. Misztal et al.: BLUPF90 and related programs (BGF90)'' (available [http://nce.ads.uga.edu/wiki/lib/exe/fetch.php?media=28-07.pdf here]). For variance component estimation, the family offers choices for simple and complicated models; see the paper ''I. Misztal: Reliable computing in estimation of variance components'' (available [http://nce.ads.uga.edu/wiki/lib/exe/fetch.php?media=reliable_computing.pdf here]). From 2009 the programs are successively modified for genomic selection using a single-step approach (or ssGBLUP) by Ignacio Aguilar and Shogo Tsuruta. ===Availability=== BLUPF90 is available in the version actual to 7/2012. ===Use=== '''Interactive use:''' * ask the scheduler for an interactive job having a single node and a single processor reserved: <source lang="bash"> $ qsub -I -l nodes=1:ppn=1 -l mem=XXXg </source> :''Note: Do not forget to apppropriately set the amount of requested memory (mem attribute) and/or another job requirements.'' * load the BLUPF90 module into the environment: <source lang="bash"> $ module add BLUPF90 </source> * run the relevant program (airemlf90, blupf90, gibbs1f90, ...) in accordance with the documentation. The list of available programs is available on the [http://nce.ads.uga.edu/wiki/doku.php?id=application_programs producer's WWW server]. <source lang="bash"> $ blupf90 </source> '''Batch use:''' * ''As far as we know, the program does not allow to be used in a batch way (passing the input file via a parameter). If you know how to run the program in a batch mode, please, let us know.'' ===Documentation=== The documentation is available on the [http://nce.ads.uga.edu/wiki/doku.php?id=documentation producer's WWW server] or locally in the <code>/software/BLUPF90/doc/</code> directory. ===License=== The programs are free for research but their use '''should be acknowledged in publications'''. ===Supported platforms=== Linux ===Program administrator=== Ludmila Zavadilová, Tomáš Rebok [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://nce.ads.uga.edu/wiki/doku.php [[Kategorie:Applications]] pqgsqg95lm7u5kj9hgi21vv516nfgjk 0 302 5046 4957 2016-05-13T09:13:03Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Tablet|Česká verze--> =Description= '''Tablet''' is a lightweight, high-performance graphical viewer for next generation sequence assemblies and alignments. =Availability= * '''Module <code>tablet-1.14</code>:''' ''Tablet version 1.14.04.10'', 64-bit * Module <code>tablet-1.13</code>: Tablet version 1.13.07.31, 64-bit =Use= module add tablet-1.13 {{Application with GUI}} =Documentation= http://bioinf.scri.ac.uk/tablet/ =License= Free for anyone =Program administrator= Petr Navrátil, Tom Rebok [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://bioinf.scri.ac.uk/tablet/ [[Category:Applications]] [[Category:English]] rsgbndut36cmtux20a9zlwem0uxul33 0 271 5052 4952 2016-05-13T09:13:09Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--GNU Parallel|Česká verze--> =Description= GNU parallel is a shell tool for executing jobs in parallel using one or more computers. =Availability= Version 20130422. Freely available to all users. =Licence= [http://www.gnu.org/licenses/gpl.html GPLv3] =Use= You can load the module via the command module add parallel This makes available the path to the program binaries and manual pages. =Documentation= Documentation is available in manual pages or at the project [http://www.gnu.org/software/parallel/man.html site]. ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.gnu.org/software/parallel/ [[Category:Applications]] [[Category:English]] 7eadehaulnfsyau6wjltisvuenwuq9d 0 327 5060 5033 2016-05-13T09:13:16Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Theano|Česká verze--> = Description = '''Theano''' is a Python library that allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. Theano features: * tight integration with NumPy * transparent use of a GPU * efficient symbolic differentiation * speed and stability optimizations = Availability = '''Theano version 0.6''': module <code>theano-0.6</code> = Use = Load the module by running: module add theano-0.6 Now you can in your python scripts call import theano = License = '''BSD license''' = Documentation = The Theano documentation is available at [http://deeplearning.net/software/theano/theano.pdf official webpage]. = Program administrator = ([mailto:meta@cesnet.cz meta@cesnet.cz]) = Homepage = URL: http://deeplearning.net/software/theano/ [[Category:Applications]] [[Category:English]] r9dwivs8kjfb59b0z50gtrj9h6mbxjc 0 269 5065 4888 2016-05-13T09:13:22Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--XCrysDen|Česká verze--> = Description = XCrySDen is a crystalline and molecular structure visualisation program aiming at display of isosurfaces and contours, which can be superimposed on crystalline structures and interactively rotated and manipulated. = Availability = Verze 1.4.1 a 1.5.53. Freely available for every user. = Licence = GNU/GPL v2, see http://www.gnu.org/licenses/gpl-2.0.html for details. = Use = Initialize module with command: module add xcrysden # for the newest version, 1.5 actually module add xcrysden-1.5 # for version 1.5 module add xcrysden-1.4 # for version 1.4 After initialization is available a program run script <tt>xcrysden</tt>. Just type xcrysden to run the program. {{Application with GUI}} = Documentation = http://www.xcrysden.org/Documentation.html = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://www.xcrysden.org [[Category:Applications]] [[Category:English]] 0w1s31ua98w0bue9sb1vaumwvyk9sm1 0 213 5068 4994 2016-05-13T09:13:25Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--SAMtools|Česká verze--> =Description= SAM Tools provide various utilities for manipulating alignments in the SAM format, including sorting, merging, indexing and generating alignments in a per-position format. =Availability= Verze 0.1.18, 0.1.19 and 1.2. Available modules: samtools-0.1.18 samtools-0.1.18alt samtools-0.1.19 samtools-0.1.19fPIC samtools-1.2 =Use= module add samtools-0.1.18 samtools for more details see Documentation. Module samtools-0.1.18alt is compiled with -fPIC. =Documentation= http://samtools.sourceforge.net/ =Licence= MIT License =Program manager= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://samtools.sourceforge.net/<br /> http://www.htslib.org/ [[Category:Applications]] [[Category:English]] ql0gijrq54kddsqburagxevus2ee0zm 0 395 5070 4907 2016-05-13T09:13:26Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--PartitionFinder|Česká verze--> =Description= PartitionFinder is free open source software to select best-fit partitioning schemes and models of molecular evolution for phylogenetic analyses. Most of the lab's coding output currently goes into PartitionFinder. =Availability= Application is available in * Module <code>partitionfinder-1.1.1</code>: Version 1.1.1 =Use= Steps to run the application <source lang="bash"> $ module add partitionfinder-1.1.1 $ PartitionFinder.py </source> You can run the application by typing <code>PartitionFinder.py</code> instead of <code>python PartitionFinder.py</code> which is used in the manual. After this command one can add other parameters, see <code>PartitionFinder.py --help</code>. =Documentation= [http://www.robertlanfear.com/partitionfinder/assets/Manual_v1.1.1.pdf Manuál 1.1.1]. =Licence= GNU General Public licence. =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= URL: http://www.robertlanfear.com/partitionfinder [[Category:Applications]] [[Category:English]] 0je68s6ynq64ci57oshfe0uneg2krun 0 332 5072 4931 2016-05-13T09:13:28Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Python - moduly|Česká verze--> =Description= Python programming language supports extensions in the form of so called 'modules'. To ease the administration of such modules we grouped them into one SW package (module). These packages are dependent on the Python version and used compiler. =Availability= Freely available to all MetaCentrum users. Module list: {|class="tabulka" |- !width="150px" |Module name !Included Python modules |- | python26-modules-gcc, python26-modules-intel | ase, backports.ssl-match-hostname, Cython, matplotlib, nose, numexpr, numpy, pandas, patsy, pip, pyparsing, pysam, python-dateutil, pytz, ruffus, scipy, setuptools, six, statsmodels, tables, tornado, wsgiref |- | python27-modules-gcc, python27-modules-intel | ase, backports.ssl-match-hostname, biopython, Cython, CVXOPT, keras, matplotlib, nose, numexpr, numpy, pandas, patsy, PICOS, pip, pyaudio, pyparsing, [https://github.com/dereneaton/pyrad/ pyrad], pysam, python-dateutil, python-igraph, python-yaml-logger, pytz, ruffus, scikit-learn, scipy, setuptools, six, statsmodels, tables, Theano, tornado, wsgiref, dendropy, lxml |- | python34-modules-gcc, python34-modules-intel | numpy, pandas, pip, python-dateutil, pytz, setuptools, six, RNFtools |- | python27-modules-intel15 | apptools, configobj, cycler, Cython, decorator, funcsigs, ipython, ipython-genutils, matplotlib, mayavi, mock, mpi4py, nose, numexpr, numpy, pandas, path.py, pbr, petsc, pexpect, pickleshare, pip, pyface, Pygments, pyparsing, python-dateutil, pytz, PyVTK, scipy, setuptools, sfepy, simplegeneric, six, sympy, tables, traitlets, traits, traitsui, VTK, wheel |- |} =====Module notes===== * You can get the module versions by command <code lang="bash">pip list</code> * To use of python module <tt>python-igraph</tt> you should also add one of the SW modules <tt>igraph-0.7.1-gcc</tt> and <tt>igraph-0.7.1-intel</tt>. * To use of python module <tt>tables</tt> you should also add one of the SW modules <tt>hdf5-1.8.12-gcc</tt> or <tt>hdf5-1.8.12-intel</tt>. =Licence= Differs according to used Python module. =Use= Initialize environment with command: module add python26-modules-intel You can also install your own Python module, see [[How to install application#Python packages|SW installation]] for details. =Documentation= Python modules documentation is available through [https://pypi.python.org/pypi PyPI installer page]. =Program manager= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= URL: http://www.python.org/ [[Category:Applications]] [[Category:English]] qdwwv980u5o93vryg7m2d2twx1wssn4 0 252 5074 4976 2016-05-13T09:13:29Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Freesurfer|Česká verze--> = Description = FreeSurfer is a set of automated tools for reconstruction of the brain’s cortical surface from structural MRI data, and overlay of functional MRI data onto the reconstructed surface. = Availability = Freely available for every user. Modules: freesurfer-5.3.0 freesurfer-5.1.0 = Licence = Licence of The General Hospital Corporation, Boston MA: http://surfer.nmr.mgh.harvard.edu/fswiki/FreeSurferSoftwareLicense = Use = Initialize module with command: module add freesurfer-5.3.0 After initialization are available all required system variables and program binaries. Variable are easily changeable through shell. {{Application with GUI}} ==Parallel usage== For parallel use you can run the job on the multiprocessor machine (with proper amount of memory) and add the proper switch to freesurfer tools which supports it. For example to the <tt>mri_glmfit-sim</tt> you can add <tt>--bg number_of_processors</tt> parameter. On the basis of [http://wikis.la.utexas.edu/imagelab/book/tutorials-group-analysis tutorial] and [ftp://surfer.nmr.mgh.harvard.edu/pub/data/buckner_data-tutorial_subjs.tar.gz downloaded data set] you can use following example: qsub -X -I -q short -l nodes=1:ppn=8,mem=4gb#excl ... export SUBJECTS_DIR=/scratch/hanousek/fsf_demo/samples/buckner_data/tutorial_subjs cd $SUBJECTS_DIR/glm mris_preproc --fsgd gender_age.fsgd --cache-in thickness.fwhm10.fsaverage --target fsaverage --hemi lh --out lh.gender_age.thickness.10.mgh mri_glmfit --y lh.gender_age.thickness.10.mgh --fsgd gender_age.fsgd dods --C lh-Avg-thickness-age-Cor.mtx --surf fsaverage lh --cortex --glmdir lh.gender_age.glmdir mri_glmfit-sim --glmdir lh.gender_age.glmdir --sim mc-z 100 4 mc-z.negative --sim-sign neg --overwrite --bg 8 Computing of 100 iterations run by the last row lasts without 8-processor parallelization (without parameter <tt>--bg 8</tt>) approx. 50 minutes, with parallelization about 8 minutes. You can use more processors to shorten the computing. ===Possible faults=== ====Memory allocation==== Memory allocation failure, ie: reading colortable from annotation file... colortable with 36 entries read (originally /autofs/space/terrier_001/users/nicks/freesurfer/average/colortable_desikan_killiany.txt) *** glibc detected *** mri_surfcluster: malloc(): memory corruption: 0x0000000029b768d0 *** Occured while try the submission: /storage/brno2/home/hanousek/inst/freesurfer_project/samples/buckner_data/tutorial_subjs/glm$ mri_glmfit-sim --glmdir lh.gender_age.glmdir --sim mc-z 40 4 mc-z.negative --sim-sign neg --overwrite after adding the <tt>--bg 8</tt> was everything OK. '''Conclusion:''' Maybe it depends on the count of executed iterations defined by <tt>--sim mc-z 40 4 mc-z.negative</tt> parameter (here 40). = Documentation = Documentation and support are at [http://surfer.nmr.mgh.harvard.edu/fswiki/FreeSurferWiki#Documentation wiki] of Freesurfer project. Valuable tutorial i available at [http://wikis.la.utexas.edu/imagelab/book/tutorials-group-analysis University of Texas]. = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://surfer.nmr.mgh.harvard.edu/ [[Category:Applications]] [[Category:English]] ht44ynpf2ixgqfsxih7tz6c5h7wwrk2 0 231 5079 4948 2016-05-13T09:13:33Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Novoalign2|Česká verze--> ===Description=== An aligner for single-ended and paired-end reads from the Illumina Genome Analyser. Novoalign finds global optimum alignments using full Needleman-Wunsch algorithm with affine gap penalties. ===Availability=== Version 2.8.2 ===Use=== module add novoalign-2.8.2 novoalign see Documentation. ===Documentation=== http://www.novocraft.com/ ===Licence=== Free for non commercial use. ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.novocraft.com/ [[Kategorie:Applications]] 9wlvuti2ucyqy8h4otmqtvy72wjb5q5 0 212 5081 4971 2016-05-13T09:13:35Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Trinity|Česká verze--> ===Description=== Trinity, developed at the Broad Institute and the Hebrew University of Jerusalem, represents a novel method for the efficient and robust de novo reconstruction of transcriptomes from RNA-seq data. Trinity combines three independent software modules: Inchworm, Chrysalis, and Butterfly, applied sequentially to process large volumes of RNA-seq reads. Trinity partitions the sequence data into many individual de Bruijn graphs, each representing the transcriptional complexity at at a given gene or locus, and then processes each graph independently to extract full-length splicing isoforms and to tease apart transcripts derived from paralogous genes. ===Availability=== Verze 2012-06-8, 2013-11-10, 2014-04-13p1 and '''2.0.6'''. Available modules (from the oldest): trinity trinity-201311 trinity-201404 trinity-2.0.6 ===Use=== module add trinity-2.0.6 Trinity --CPU 2 for run on 2 processors. For details see Documentaion. Perl libriraries, plugins etc. are saved in /software/trinity or /software/trinity/VERSION. trinity-2.0.6 '''requires jdk-7''' (not work with jdk-8), therefore be careful with adding other modules (ideally, add trinity module as the last one). ===Documentation=== http://trinityrnaseq.sourceforge.net/ ===Licence=== See /software/trinity/LICENSE ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://trinityrnaseq.sourceforge.net/ ===Know how=== To combine with R, apply module R-3.1.1. [[Kategorie:Applications]] [[Category:English]] m5g2t1nd8lzu8xkm5j1jvrzj4bu61d6 0 243 5082 4782 2016-05-13T09:13:36Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Scilab|Česká verze--> =Description= '''Scilab''' (created by French scientific institutions INRIA and ENPC) is a freeware program for numerical calculations similar to MATLAB - the program allows to perform both simple arithmetics (multiplications) as well as intensive computing. For example, Scilab is used for statistical analysis, image processing, simulation of physical and chemical phenomena, etc. =Availability= * '''Module <code>scilab-5.4.0</code>:''' ''Scilab version 5.4.0'', 64-bit version, compiled with the Intel MKL support. * '''Module <code>scilab-5.5.1</code>:''' ''Scilab version 5.5.1'', 64-bit version. =Use= {{Application with GUI}} ===1. Running the application &ndash; ''Interactive mode''=== * ask the scheduler for an interactive job having a single node and a desired number of processors (<code>ppn</code> attribute) reserved <source lang="bash"> $ qsub -I -l nodes=1:ppn=X -l mem=Yg </source> :''Note: Do not forget to apppropriately set the amount of requested memory (<code>mem</code> attribute) and/or another job requirements.'' :''Note 2: If you intend to use the Scilab via its '''graphical interface''', log in the frontend tunneling the display and add the '''"<code>-X</code>" parameter''' to the <code>qsub</code> command.'' * load the Scilab module into the environment <source lang="bash"> $ module add scilab-5.4.0 </source> * run Scilab ** running the '''command-line interface''': <source lang="bash"> $ scilab-cli </source> :* running the '''graphical interface''': <source lang="bash"> $ scilab </source> ===2. Running the application &ndash; ''Batch mode:''=== * prepare the job description script -- use a general skeleton supplemented by the following lines: <source lang="bash"> ... # load the application module module add scilab-5.4.0 # run the computation scilab-cli -f input_file >output_file ... </source> * pass the job description file to the scheduler together with (at least) the requested number of processors and requested amount of memory <source lang="bash"> $ qsub -l nodes=1:ppn=X -l mem=Yg mydescriptionscript.sh </source> =Documentation= http://www.scilab.org/support/documentation =Licence= open-source, under the terms of [http://www.cecill.info/index.en.html CeCILL licence] (compatible with [http://www.gnu.org/licenses/lgpl-3.0.en.html GNU Lesser General Public License], abiding by the rules of distribution of free software). =Program administrator= Tom Rebok [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://www.scilab.org [[Kategorie:Applications]] [[Category:English]] 5hjbe6jkuxniqw6x4i7k324grmnt86q 0 187 5084 4834 2016-05-13T09:13:37Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Scientific_Python|Česká verze--> =Description= ScientificPython is a collection of Python modules that are useful for scientific computing. In this collection you will find modules that cover basic geometry (vectors, tensors, transformations, vector and tensor fields), quaternions, automatic derivatives, (linear) interpolation, polynomials, elementary statistics, nonlinear least-squares fits, unit calculations, Fortran-compatible text formatting, 3D visualization via VRML, and two Tk widgets for simple line plots and 3D wireframe models. There are also interfaces to the netCDF library (portable structured binary files), to MPI (Message Passing Interface, message-based parallel programming), and to BSPlib (Bulk Synchronous Parallel programming). =Availability= Program is currently available in version 2.9.0 with different compilations. Available modules: scientificpython-py2.6 scientificpython-py2.6-mpi scientificpython-py2.6-old scientificpython-py2.5 Version py2.6 differs from py2.6-old by compilation on Deb7 (instead of Deb6) and with support of [[NetCDF application|NetCDF]]. Versions py2.6-mpi and py2.6-old support parallel run. =Use= Load module example: module add scientificpython-py2.6 "-py-2.x" is determining Python version with which is module build. Module "load" automatically all others required files. {{MPI_application}} ===Example of running=== <source lang="bash"> skirit$ module add scientificpython-py2.6 skirit$ mpirun -mca btl ^gm -np 2 mpipython -V Python 2.6.2 </source> By option "-mca btl ^gm", warnings about unavailable myrinet (at machine without myrinet) will be switched off. =Documentation= [http://dirac.cnrs-orleans.fr/ScientificPython/ScientificPythonManual/ On-line] manual is available =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= URL: http://dirac.cnrs-orleans.fr/plone/software/scientificpython [[Kategorie:Applications]] [[Kategorie:English]] s3f4uv6d1cmxt43cio59itorjb3c2rk 0 179 5090 4778 2016-05-13T09:13:42Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--GCC|Česká verze--> ===Description=== The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Java, and Ada, as well as libraries for these languages (libstdc++, libgcj,...). ===Availability=== Program is actually available in version demon 4.3.2, 4.3.3 a 4.5.2, 4.7.0, 4.8.1, 4.8.4. ===Use=== Load module ("module ad gcc-4.8.4") and run program. ===Documentation=== [http://gcc.gnu.org/onlinedocs/ On-line] documentation is available. ===Supported platforms=== Linux ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://gcc.gnu.org/ [[Category:Applications]] [[Category:English]] 684sgse2xp2pggk4crewvtesqnoq6fv 0 352 5091 4972 2016-05-13T09:13:42Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--GPAW|Česká verze--> =Description= GPAW (Grid-based projector-augmented wave method) is a density-functional theory (DFT) Python code based on the projector-augmented wave (PAW) method and the atomic simulation environment (ASE). It uses real-space uniform grids and multigrid methods or atom-centered basis-functions. Read more about its features and the algorithms used. =Availability= Version 0.10. Freely available to all users. Modules: gpaw-0.10-py27-gcc =Licence= [http://www.gnu.org/copyleft/gpl.html GPL v3] =Use= You can load the module via the command module add gpaw-0.10-py27-gcc This makes available the path to the program binaries, libraries and manual pages. {{MPI application}} =Documentation= Documentation is available in manual pages or on https://wiki.fysik.dtu.dk/gpaw/documentation/documentation.html.. ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== https://wiki.fysik.dtu.dk/gpaw/index.html [[Category:Applications]] [[Category:English]] 18nrdild5hn2wajsq936pb7uac4p8c2 0 154 5093 4973 2016-05-13T09:13:44Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--MOPAC|Česká verze--> ===Description=== MOPAC (Molecular Orbital PACkage) is a semiempirical quantum chemistry program based on Dewar and Thiel's NDDO approximation. [http://openmopac.net/MOPAC2009.html MOPAC2009] is MOPAC2007 plus the MOZYME function. MOZYME is a linear-scaling SCF method that allows systems of up to 15,000 atoms to be modeled. Optimize systems of up to 15,000 atoms, e.g. proteins *NEW PM6 parameterization from experimental & ab initio *More accurate* heats of formation and geometries *ALL main group elements & transition metals *Serious errors from PM3 and AM1 corrected *Crystals, surfaces & polymers with periodic boundaries *FREE to academics ===Availability=== Program is actually available in version MOPAC2009 and MOPAC2012. It is required to [https://perun.metacentrum.cz/metavoint/?r=%2ffed%2fregistrar%2f%3fvo%3dmeta%26locale%3dcs%26group%3dlic_mopac accept] non-commercial use of the software. Available modules: mopac2012 mopac2012-gpu mopac2009 ===Use=== Load module (module ad mopac2009) and run program. ===Documentation=== [http://openmopac.net/ On-line] documentation is available. ===Supported platforms=== Linux ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://openmopac.net/ <br /> page for academic users: http://openmopac.net/index.html [[Kategorie:Applications]] [[Kategorie:English]] 6kob8743xg4mxpuc6dvqio6hqztcrp8 0 297 5092 5023 2016-05-13T09:13:43Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Seqtk|Česká verze--> = Description = Seqtk is a fast and lightweight tool for processing sequences in the FASTA or FASTQ format. It seamlessly parses both FASTA and FASTQ files which can also be optionally compressed by gzip. = Availability = Version 1.0 = Licence = Not specified. = Use = module add seqtk-1.0 seqtk = Documentation = https://github.com/lh3/seqtk = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = https://github.com/lh3/seqtk [[Category:Applications]] [[Category:English]] nfmuqf5h5zayufht18vjaw4tsosr5mv 0 229 5094 5086 2016-05-13T09:13:45Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--WGS|Česká verze--> ===Description=== Celera Assembler (CA) is a whole-genome shotgun (WGS) assembler for the reconstruction of genomic DNA sequence from WGS sequencing data. ===Availability=== Version 7.0 and 8.1. Available modules: wgs-7.0 wgs-8.1 ===Use=== module add wgs-8.1 (or wgs-7.0) chimera see Documentation ===Documentation=== http://sourceforge.net/projects/wgs-assembler/ ===Licence=== GPL ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://sourceforge.net/projects/wgs-assembler/ [[Kategorie:Applications]] [[Kategorie:English]] ktc0idkvsdb507acplb8rx8i5agiscm 0 355 5095 5064 2016-05-13T09:13:46Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--QT|Česká verze--> = Description = Qt is a cross-platform application and UI framework for developers using C++ or QML, a CSS & JavaScript like language. = Availability = Version 4.8.5. Freely available for every user. = Licence = [http://qt-project.org/doc/qt-4.8/licensing.html Various], depending on used component. = Use = Initialize module with command: module add qt-4.8.1 {{Application with GUI}} = Documentation = Documentation for QT 4.8 is available [http://qt-project.org/doc/qt-4.8/ on-line]. = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://qt-project.org/ [[Category:Applications]] [[Category:English]] 96m8pbdgjrfamhdzyutlfvxjy8ml9m4 0 199 5096 5016 2016-05-13T09:13:47Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--SOAPdenovo|Česká verze--> = Description = '''SOAPdenovo''' is a short-read assembler designed for handaling reads from NGS sequencing technology that allows to sequence even big-sized genomes like human genome. The program is specially designed to assemble Illumina GA short reads. = Availability = * '''version 2.0 r240''' (module <code>SOAPdenovo2</code>) -- SOAPdenovo2 together with other related tools (namely SOAPec, GapCloser, Data prepare and Error Correction modules) * ''version 1.05'' (module <code>SOAPdenovo</code>) -- SOAPdenovo version 1.05 = Usage = Load the module (<code>module add SOAPdenovo2</code>) and use the available tools (<code>Corrector_AR, Corrector_HA, ErrorCorrection, finalFusion, GapCloser, KmerFreq_AR, KmerFreq_HA, SOAPdenovo-127mer, SOAPdenovo-63mer</code>) in accordance with the program documentation. = Documentation = Documentation is available on official website. Help and instruction manual is also a part of the package. = Supported platforms = Linux x86_64 = Program administrator = * Jan Ištvánek -- Laboratory of molecular plant genetics, PřF MU (janites at mail.muni.cz, i.jan at centrum.cz) * Tomáš Rebok (rebok at ics.muni.cz) = Homepage = URL: http://soap.genomics.org.cn/soapdenovo.html [[Kategorie:Applications]] bacnqwcgodjko8cnxluoghprfw3k9ge 0 191 5097 4903 2016-05-13T09:13:49Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--ffmpeg|Česká verze--> ===Description=== Module ffmpeg contains application for conversion of audio and video files (binary executable "ffmpeg") and C++ libraries for ffmpeg usage within your applications. Module ffmpeg is used by [[OpenCV_application]] module. ===Availability=== Current version of ffmpeg in MetaCentrum: 1.0 (stable version, compiled 1.10.2012), available in module '''ffmpeg'''. Compiled with support of external codecs: libx264, libmp3lame, libfaac ===Use=== Initialize environment: <source lang="bash"> module add ffmpeg </source> ===Documentation=== [http://ffmpeg.org/ Official on-line ffmpeg documentation] ===Licence=== LGPL or GPL (depends on compilation options) ===Supported platforms=== Linux (in MetaCentrum environment), generally Windows and MacOS too ===Program manager=== [http://www.kky.zcu.cz/en/campr-pavel Pavel Campr] ===Homepage=== [http://ffmpeg.org/ ffmpeg homepage] [[Kategorie:Applications]] g8uiazwoeftu0u20vlp5gt1nu1bewl5 0 366 5098 5032 2016-05-13T09:13:51Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--SNAP|Česká verze--> = Description = SNAP is a new sequence aligner that is 3-20x faster and just as accurate as existing tools like BWA-mem, Bowtie2 and Novoalign. It runs on commodity x86 processors, and supports a rich error model that lets it cheaply match reads with more differences from the reference than other tools. This gives SNAP up to 2x lower error rates than existing tools (in some cases) and lets it match larger mutations that they may miss. SNAP also natively reads BAM, FASTQ, or gzipped FASTQ, and natively writes SAM or BAM, with built-in sorting, duplicate marking, and BAM indexing. = Availability = Version 1.0.17 = Licence = Apache 2 = Use = module add snap-1.0.17 snap unit_tests = Documentation = http://snap.cs.berkeley.edu/#documentation = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://snap.cs.berkeley.edu/ [[Category:Applications]] [[Category:English]] m5jqiyi9n9ruf7e59d3x7mwek2grjwz 0 244 5101 4985 2016-05-13T09:13:55Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--IDV|Česká verze--> = Description = '''IDV''' is a geoscience display and analysis software system with many of the standard data displays that other Unidata software (e.g. GEMPAK and McIDAS) provide. It brings together the ability to display and work with satellite imagery, gridded data (for example, numerical weather prediction model output), surface observations, balloon soundings, NWS WSR-88D Level II and Level III RADAR data, and NOAA National Profiler Network data, all within a unified interface. It also provides 3-D views of the earth system and allows users to interactively slice, dice, and probe the data, creating cross-sections, profiles, animations and value read-outs of multi-dimensional data sets. = Availability = ''Integrated Data Viewer (IDV)'' is available in the '''version 3.1''' (via module <code>idv-3.1</code>) = Use = * tunnelling the graphical display, log into a MetaCentrum frontend: <source lang="bash"> $ ssh -X skirit.metacentrum.cz </source> * run the application either locally on the frontend or (better) on a computing node: ** ''running on a computing node:'' *** ask the scheduler for an interactive job -- specify the number of processors you intend to use (the "<code>Y</code>" attribute), as well the amount of memory necessary (the "<code>Z</code>" attribute). Do not forget to tunnel the graphical display again via the "<code>-X</code>" option: <source lang="bash"> $ qsub -I -X -l nodes=1:ppn=Y -l mem=Zg </source> ::* load the application module and run the IDV: <source lang="bash"> skirit$ module add idv-3.1 skirit$ runIDV # or: simpleIDV </source> :* ''running on a frontend:'' ::* load the application module and run the IDV: <source lang="bash"> skirit$ module add idv-3.1 skirit$ runIDV # or: simpleIDV </source> {{Application with GUI}} = Documentation = The IDV documentation is available on the [http://www.unidata.ucar.edu/software/idv/docs/userguide/toc.html producer's webpage]. = Program administrator = * Tomáš Rebok (rebok at ics.muni.cz) = Homepage = URL: http://www.unidata.ucar.edu [[Category:Applications]] [[Category:English]] ru1w42edolaurhu5uw3808x1bdchdwy 0 337 5102 4864 2016-05-13T09:13:56Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--HMMER|Česká verze--> =Description= HMMER is used for searching sequence databases for homologs of protein sequences, and for making protein sequence alignments. It implements methods using probabilistic models called profile hidden Markov models (profile HMMs). =Availability= Verze 3.1b1. Freely available to all users. Modules: hmmer-3.1b1-gcc hmmer-3.1b1-intel =Licence= [http://www.gnu.org/copyleft/gpl.html GPL v3] =Use= You can load the module via the command module add hmmer-3.1b1-gcc This makes available the path to the program binaries, libraries and manual pages. =Documentation= Documentation is available in manual pages or on http://hmmer.janelia.org/help. ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://hmmer.janelia.org/ [[Category:Applications]] [[Category:English]] 240jv47lr6klaghnda9isiemdw7lgve 0 232 5104 4755 2016-05-13T09:13:57Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Fastx|Česká verze--> ===Description=== The FASTX-Toolkit is a collection of command line tools for Short-Reads FASTA/FASTQ files preprocessing. ===Availability=== Version 0.0.13 ===Use=== module add fastx-0.0.13 fastx_trimmer see Documentation ===Documentation=== http://hannonlab.cshl.edu/fastx_toolkit/commandline.html ===Licence=== GNU Affero GPL, version 3 ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://hannonlab.cshl.edu/fastx_toolkit [[Kategorie:Applications]] gbea4sm4h21ib5j285jid2ojl6v24qk 0 335 5106 4967 2016-05-13T09:13:59Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--UEA_sRNA_Workbench|Česká verze--> = Description = The UEA sRNA workbench is a simple to use, downloadable sRNA software package based on algorithms developed for the original ''UEA sRNA Toolkit'' that will perform a complete analysis of single or multiple-sample small RNA datasets from both plants and animals to identify interesting landmarks (such as detection of novel micro RNA sequences) or other tasks such as profiling small RNA expression patterns in genetic data. = Availability = '''UEA sRNA workbench version 3.0''': module <code>sRNAworkbench3.0</code> = Use = Load the relevant module and run the UEA sRNA workbench application. The application requires a graphical interface. <source lang=bash> $ module add sRNAworkbench3.0 $ start_sRNAworkbench3.0 </source> Tools that the UEA sRNA workbench consists of can be used also separately from command line, though. In order to do that, you'll probably need to load module to run Java (which is otherwise added by start_sRNAworkbench3.0 starting script. <source lang=bash> $ module add jdk-7 # load Java module $ java -jar /software/sRNAworkbench3.0/srna-workbenchV3.0_L/Workbench.jar -tool <toolname> </source> Example of starting 'mircat' tool: <source lang=bash> $ module add jdk-7 # load Java module $ java -jar /software/sRNAworkbench3.0/srna-workbenchV3.0_L/Workbench.jar -tool mircat -srna_file <srna-file> -genome <genome-file> [-params <params-file>] </source> {{Application with GUI}} = Documentation = The UEA sRNA workbench documentation is available on the [http://srna-workbench.cmp.uea.ac.uk/tutorialsmain/ official producer's webpage] = Homepage = URL: http://srna-workbench.cmp.uea.ac.uk [[Category:Applications]] [[Category:English]] lndggprs4ny4xhl194lcbuaeep4k14f 0 325 5109 4743 2016-05-13T09:14:01Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--HDF5|Česká verze--> = Description = HDF5 is a data model, library, and file format for storing and managing data. It supports an unlimited variety of datatypes, and is designed for flexible and efficient I/O and for high volume and complex data. The HDF5 Technology suite includes tools and applications for managing, manipulating, viewing, and analyzing data in the HDF5 format. = Availability = Versions 1.8.4, 1.8.5 a 1.8.12. Freely available to all users. Modules: * Without MPI support hdf5-pgi hdf5-1.8.4p1 (gcc compilation) * With OpenMPI support hdf5-1.8.12-gcc hdf5-1.8.12-intel hdf5-1.8.12-pgi = Licence = [http://www.hdfgroup.org/ftp/HDF5/current/src/unpacked/COPYING BSD-style licence] = Use = Initialize environment with command: module add hdf5-1.8.12-intel {{MPI application}} = Documentation = Documentation is at http://www.hdfgroup.org/documentation. = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://www.hdfgroup.org/HDF5/ [[Category:Applications]] [[Category:English]] bgu8rnv8iqxrt541zweonqjw1vaul96 0 581 5110 5009 2016-05-13T09:14:01Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--CS-Rosetta|Česká verze--> ===Description=== Chemical-Shift-ROSETTA (CS-ROSETTA) is a robust protocol for de novo protein structure generation, using as input parameters the 13CA, 13CB, 13C', 15N, 1HA and 1HN NMR chemical shifts. These shifts are generally available at the early stage of the traditional NMR structure determination procedure, prior to the collection and analysis of structural restraints. The CS-ROSETTA approach, as shown below, utilizes SPARTA-based selection of protein fragments from the PDB, in conjunction with a regular ROSETTA Monte Carlo assembly and relaxation procedure. ===Use=== Initialize enviroment (module add csrosetta) and run csrosetta. ===Documentation=== Documentatio n is available on http://www.csrosetta.org/manual (http://spin.niddk.nih.gov/bax/software/CSROSETTA/index.html) of producer or locally in directory /software/rosetta-2.3.0/doc. ===Licence=== User have to agree with a [http://metavo.metacentrum.cz/en/myaccount/licence.html licence]. ===Supported platforms=== Linux ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://www.csrosetta.org/ http://www.rosettacommons.org/ [[Kategorie:Applications]] 4bd86cyk1wzv2qe4ba7pb8io27c01of 0 260 5111 4812 2016-05-13T09:14:02Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--NCO|Česká verze--> ===Description=== NCL is an interpreted language designed specifically for scientific data analysis and visualization. Supports netCDF3/4, GRIB1/2, HDF-SDS, HDF4-EOS, binary, shapefiles and ascii files. ===Availability=== There is version 4.2.5 on MetaCentrum machines compiled by gcc with MPICH-3.0.2 support. ===Licence=== Built from several licences, see [http://www.earthsystemgrid.org/browse/viewLicense.htm?licenseId=52b2d37a-d902-431e-9092-0413e36c91fd] for details. Freely available on MetaCentrum machines. ===Use=== Initialize module with command module add nco-4.2.5-gcc After initialization are correctly set up the following system variables: $PATH, $NCARG_ROOT ===Documentation=== http://www.ncl.ucar.edu/Document/ ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://nco.sourceforge.net/ [[Category:Applications]] naj63hskzqliosf1k313psqsz4ewxp9 0 157 5119 4990 2016-05-13T09:14:09Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Tinker|Česká verze--> ===Description=== The TINKER molecular modeling software is a complete and general package for molecular mechanics and dynamics, with some special features for biopolymers. ===Availability=== Program is actually available in version 5.0 ===Use=== Load module ("module ad tinker") ===Documentation=== [http://dasher.wustl.edu/tinkerwiki TINKER Wiki Site] [http://dasher.wustl.edu/tinker/downloads/guide.pdfTINKER User's guide] ===Licence=== User have to agree with [http://metavo.metacentrum.cz/osobni/personal/eula.do licence] ===Supported platforms=== Linux ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://dasher.wustl.edu/tinker/ [[Kategorie:Applications]] 4u039xbi5gvttfb4twv8hxxabehtx5n 0 375 5118 5063 2016-05-13T09:14:08Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--FastTree|Česká verze--> ===Description=== FastTree infers approximately-maximum-likelihood phylogenetic trees from alignments of nucleotide or protein sequences. FastTree can handle alignments with up to a million of sequences in a reasonable amount of time and memory. For large alignments, FastTree is 100-1,000 times faster than PhyML 3.0 or RAxML 7. ===Availability=== FastTree 2.1.8 ===Use=== module add fasttree-2.1.8 fasttreemp ===Documentation=== http://meta.microbesonline.org/fasttree/#Install ===Licence=== ??? ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://meta.microbesonline.org/fasttree/#Install [[Kategorie:Applications]] 3xx0oexwqynv8rsd61bxksytbr6kn9r 0 287 5122 4732 2016-05-13T09:14:11Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Xmipp|Česká verze--> =Description= Xmipp is a suite of image processing programs, primarily aimed at single-particle 3D electron microscopy. =Availability= Program is actually available in version 3.0.1. Module versions: xmipp-3.0.1 =Licence= [http://www.gnu.org/licenses/gpl-2.0.html GNU/GPL v2 licence] + citation request: ''If you have used Xmipp in your work, please cite our papers as given in the [http://xmipp.cnb.csic.es/twiki/bin/view/Xmipp/ListOfReferences List Of References].'' =Use= Initialize environment by command module add xmipp-3.0.1 This makes available the path to program binaries as well as aliases for the most used ones: <source lang="bash"> alias xa='xmipp_apropos' alias xb='xmipp_browser' alias xic='xmipp_image_convert' alias xih='xmipp_image_header' alias xio='xmipp_image_operate' alias xis='xmipp_image_statistics' alias xmipp='xmipp_protocols' alias xmipp_show='xmipp_showj' alias xmp='xmipp_metadata_plot' alias xmu='xmipp_metadata_utilities' alias xp='xmipp_protocols' alias xrf='xmipp_resolution_fsc' alias xrs='xmipp_resolution_ssnr' alias xs='xmipp_showj' alias xsj='xmipp_showj' alias xtf='xmipp_transform_filter' alias xtg='xmipp_transform_geometry' alias xtn='xmipp_transform_normalize' </source> {{Application with GUI}} =Documentation= User documentation is available in http://xmipp.cnb.csic.es/twiki/bin/view/Xmipp/Documentation. =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://xmipp.cnb.csic.es/twiki/bin/view/Xmipp/WebHome [[Category:Applications]] [[Category:English]] 0iengezmysguuxrk6oud1knafez5d7d 0 190 5125 4939 2016-05-13T09:14:14Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Lisp-CMUCL|Česká verze--> ===Description=== CMUCL is a free implementation of the Common Lisp programming language which runs on most major Unix platforms. It mainly conforms to the ANSI Common Lisp standard. ===Availability=== Program is actually available in version 19f ===Use=== Load module ("module ad list-cmucl-19f") ===Documentation=== http://www.cons.org/cmucl/ ===Supported platforms=== Linux ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.cons.org/cmucl/ [[Kategorie:Applications]] 0gzrfwc3ho61xvwhs0bgk0q8katilyw 0 177 5126 4926 2016-05-13T09:14:14Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--OpenFOAM|Česká verze--> ===Description=== OpenFOAM® (Open Field Operation and Manipulation) CFD Toolbox can simulate anything from complex fluid flows involving chemical reactions, turbulence and heat transfer, to solid dynamics, electromagnetics and the pricing of financial options. ===Licence=== [http://www.gnu.org/copyleft/gpl.html GNU/GPL] ===Availability=== Program is actually available in version 1.5 and 2.3.0. ===Use=== Load modul module add openfoam-2.3.0 If you want to use the job monitoring you must set the variable FOAM_JOB_DIR so that it points to your directory which must contain two subdirectories named "runningJobs" and "finishedJobs". OpenFOAM use this directories to store the data about jobs. You can get this information about your job via "foamCheckJobs" or "foamPrintJobs" command. ===Documentation=== http://www.openfoam.org/docs/ ===Supported platforms=== Linux ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://www.opencfd.co.uk/openfoam/index.html#openfoam [[Kategorie:Applications]] [[Kategorie:English]] lska8fn7sy5zjjkz2pel88csy7bb3u5 0 251 5127 4839 2016-05-13T09:14:16Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--FMRIB Software Library (FSL)|Česká verze--> === Description === FSL is a comprehensive library of analysis tools for FMRI, MRI and DTI brain imaging data. === Availability === Version 4.1.4, 5.0.1 and 5.0.6. Freely available for every user. Available modules: fsl (fsl-4.1.4) fsl-5.0.1 fsl-5.0.6 === Licence === before using FSL in the MetaCentrum, you have to [http://metavo.metacentrum.cz/en/myaccount/licence.html agree with the licence]. === Use === Initialize module with command: module add fsl-5.0.1 After initialization are available all required system variables and program binaries. Variable are easily changeable through shell. Parallel usage of some tools can be accomplished by deassembling data into smaller units. After computation of each unit can be the results composed together. === Documentation === Documentation and support are at wiki of FSL project: http://fsl.fmrib.ox.ac.uk/fsl/fslwiki/Support === Program manager === [mailto:meta@cesnet.cz meta@cesnet.cz] === Homepage === http://fsl.fmrib.ox.ac.uk/fsl/fslwiki/ [[Category:Applications]] b4qirdcyciv757rwsh3qb09sksy7pw7 0 323 5128 5066 2016-05-13T09:14:17Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--GMAP (GSNAP)|Česká verze--> = Description = * GMAP: A Genomic Mapping and Alignment Program for mRNA and EST Sequences * GSNAP: Genomic Short-read Nucleotide Alignment Program = Availability = Version from May 2014. Freely available to users. Modules: gmap-2013-11-27 gmap-2014-05-06 = Licence = Own, freely available to users except commercial packaging. = Use = Initialize environment with command: module add gmap-2013-11-27 = Documentation = Basic documentation is available in file <tt>/software/gmap-2013-11-27/doc/README</tt>. = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://research-pub.gene.com/gmap/ [[Category:Applications]] [[Category:English]] kf8wifeyvy17lmhjwejepk1pw9iqnud 0 300 5131 4810 2016-05-13T09:14:20Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--MAFFT|Česká verze--> ===Description=== '''MAFFT''' is a fast multiple sequence alignment program for aminoacids and nucleotids. According to number of sequences and demands to time and accuracy it offers a range of multiple alignment methods, L-INS-i (accurate; for alignment of <∼200 sequences), FFT-NS-2 (fast; for alignment of <∼10,000 sequences), etc. ===Availability=== '''Module <code>mafft-7.029</code>:''' ''MAFFT version 7.029'', 64-bit ===Use=== module add mafft-7.029 mafft [arguments] input > output ===Documentation=== http://mafft.cbrc.jp/alignment/software/manual/manual.html ===License=== Free for academic users ===Supported platforms=== amd64 ===Program administrator=== Petr Navrátil, Tom Rebok [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://mafft.cbrc.jp/alignment/software/ [[Kategorie:Applications]] [[Category:Applications]] souycoobekcj94gbcwxsxi3e6xo66bo 0 281 5134 4943 2016-05-13T09:14:22Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Argus|Česká verze--> =Description= Argus is the network Audit Record Generation and Utilization System. The Argus Project is focused on developing network activity audit strategies and prototype technology to support Network Operations, Performance and Security Management. If you look at packets to solve problems, or you need to know what is going on in your network, right now or way back then, you should find Argus a useful tool. =Availability= Version 3.0.6.1. Freely available for all users. =Licence= [http://www.gnu.org/licenses/old-licenses/gpl-2.0.html GNU/GPL v2 licence] =Use= module add argus-3.0.6 After the module load all paths to necessary binaries are available. =Documentation= Documentation for available commands are available in manual pages. More detailed program documentation is at http://qosient.com/argus/manuals.shtml. =Program manager= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= http://qosient.com/argus/ [[Category:Applications]] [[Category:English]] qiiay9z9ul68jx079lxrtrupulitxck 0 220 5135 4816 2016-05-13T09:14:23Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Emboss|Česká verze--> ===Description=== EMBOSS is a free Open Source software analysis package specially developed for the needs of the molecular biology (e.g. EMBnet) user community. The software automatically copes with data in a variety of formats and even allows transparent retrieval of sequence data from the web. Also, as extensive libraries are provided with the package, it is a platform to allow other scientists to develop and release software in true open source spirit. EMBOSS also integrates a range of currently available packages and tools for sequence analysis into a seamless whole. EMBOSS breaks the historical trend towards commercial software packages. Within EMBOSS you will find around hundreds of programs (applications) covering areas such as: * Sequence alignment, * Rapid database searching with sequence patterns, * Protein motif identification, including domain analysis, * Nucleotide sequence pattern analysis---for example to identify CpG islands or repeats, * Codon usage analysis for small genomes, * Rapid identification of sequence patterns in large scale sequence sets, * Presentation tools for publication, and much more. ===Availability=== Version 6.5.7 ===Use=== module add emboss-6.5.7 embossdata or other binary (see Documentation). ===Documentation=== http://emboss.sourceforge.net/docs/ ===Licence=== GNU GPL ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://emboss.sourceforge.net/what/ [[Kategorie:Applications]] rxem9468y1sax9s9e7tk3sqawlfdiip 0 273 5137 4865 2016-05-13T09:14:24Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--CLUMPP|Česká verze--> =Description= CLUMPP is a program that deals with label switching and multimodality problems in population-genetic cluster analyses. CLUMPP permutes the clusters output by independent runs of clustering programs such as structure, so that they match up as closely as possible. The user has the option of choosing one of three algorithms for aligning replicates, with a tradeoff of speed and similarity to the optimal alignment. =Availability= Version 1.1.2. Freely available to all users. =Licence= ? =Use= You can load the module via the command module add clumpp This makes available the path to the program binary. To use the program type: $ CLUMPP <paramfile> =Documentation= Documentation is available in file <tt>/software/clumpp-1.1.2/CLUMPP_Manual.pdf</tt> or at the project [http://www.stanford.edu/group/rosenberglab/clumpp.html website]. ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.stanford.edu/group/rosenberglab/clumpp.html [[Category:Applications]] [[Category:English]] mkcxdd6jpvnsjtws2oadkckwmasu3ia 0 247 5138 5031 2016-05-13T09:14:25Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--BWA|Česká verze--> ===Description=== '''BWA (Burrows-Wheeler Alignment)''' tool is an efficient program that aligns relatively short nucleotide sequences against a long reference sequence such as the human genome. It implements two algorithms, '''''bwa-short''''' and '''''BWA-SW'''''. The former works for query sequences shorter than 200bp and the latter for longer sequences up to around 100kbp. Both algorithms do gapped alignment. They are usually more accurate and faster on queries with low error rates. ===Availability=== * '''Module <code>bwa-0.7.3a</code>:''' ''BWA version 0.7.3a'', 64-bit, parallel version * Module <code>bwa-0.6.2</code>: ''BWA version 0.6.2'', 64-bit, parallel version ===Use=== '''1. Running the application -- ''Interactive mode:''''' * ask the scheduler for an interactive job having a single node and a desired number of processors (<code>ppn</code> attribute) reserved <source lang="bash"> $ qsub -I -l nodes=1:ppn=X -l mem=Yg </source> :''Note: Do not forget to apppropriately set the amount of requested memory (<code>mem</code> attribute) and/or another job requirements.'' * load the Moses module <source lang="bash"> $ module add bwa-0.7.3a </source> * prepare the configuration and the input files and run the computation <source lang="bash"> $ bwa [index|samse|sampe] <options> # just single-threaded versions $ bwa [aln|bwasw] -t $PBS_NUM_PPN <options> # ALN and BWASW support multi-threaded/parallel processing </source> '''2. Running the application -- ''Batch mode:''''' * prepare the job description script -- use a general skeleton supplemented by the following lines: <source lang="bash"> ... # load the BWA module module add bwa-0.7.3a # run the computation bwa [index|samse|sampe] <options> # just single-threaded versions bwa [aln|bwasw] -t $PBS_NUM_PPN <options> # ALN and BWASW support multi-threaded/parallel processing ... </source> * pass the job description file to the scheduler together with (at least) the requested number of processors and requested amount of memory <source lang="bash"> $ qsub -l nodes=1:ppn=X -l mem=Yg mydescriptionfile.sh </source> ===Documentation=== * [http://bio-bwa.sourceforge.net/bwa.shtml online at producer's webpage] * locally: <code>$ man bwa</code> ===Licence=== The full BWA package is distributed under [http://www.gnu.org/licenses/gpl.html GNU General Public License version 3] as it uses source codes from BWT-SW which is covered by GPL. Sorting, hash table, BWT and IS libraries are distributed under the [http://opensource.org/licenses/MIT MIT license]. * '''If you use the short-read alignment component, please cite the following paper:''' ** ''Li H. and Durbin R. (2009) Fast and accurate short read alignment with Burrows-Wheeler transform. Bioinformatics, 25, 1754-60. [PMID: 19451168]'' * '''If you use the long-read component (BWA-SW), please cite:''' ** ''Li H. and Durbin R. (2010) Fast and accurate long-read alignment with Burrows-Wheeler transform. Bioinformatics. [PMID: 20080505]'' ===Supported platforms=== amd64 ===Program administrator=== Tom Rebok [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://bio-bwa.sourceforge.net/ [[Kategorie:Applications]] fn1gi4tldvp1u6ortjcv98iffax1g5s 0 155 5140 4748 2016-05-13T09:14:27Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--NAMD|Česká verze--> ===Description=== NAMD, recipient of a 2002 Gordon Bell Award, is a parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems. Based on Charm++ parallel objects, NAMD scales to hundreds of processors on high-end parallel platforms and tens of processors on commodity clusters using gigabit ethernet. NAMD uses the popular molecular graphics program VMD for simulation setup and trajectory analysis, but is also file-compatible with AMBER, CHARMM, and X-PLOR. ===Availability=== The application is available in the following versions: * '''2.10''' (module <code>namd-2.10</code> for CPU version and module <code>namd-2.10-gpu</code> for GPU version) * ''2.7'' (module <code>namd-2.7</code>) * ''2.7b1'' (module <code>namd-2.7</code>) ===Documentation=== [http://www.ks.uiuc.edu/Research/namd/current/ug/ NAMD 2.10 ] [http://www.ks.uiuc.edu/Research/namd/wiki/ NAMD Wiki] ===Licence=== User have to agree with [http://metavo.metacentrum.cz/osobni/personal/eula.do licence] ===Supported platforms=== Linux ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.ks.uiuc.edu/Research/namd/ [[Kategorie:Applications]] 1ev8dlajukbalm0oaii97izmeeft4yr 0 333 5142 5059 2016-05-13T09:14:28Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Perl - moduly|Česká verze--> =Description= Perl programming language supports extensions in the form of so called 'modules'. To ease the administration of such modules we grouped them into one SW package (module). These packages are dependent on the Perl. =Availability= Freely available to all MetaCentrum users. Module list: {|class="tabulka" |- !Module name !Included Perl modules |- | bioperl-1.6.1 | Ace (1.92), Acme::Damn (0.05), Algorithm::Munkres (0.08), App::cpanminus (1.7001), Archive::Zip (1.37), Array::Compare (2.02), BerkeleyDB (0.54), Bio (), Bio::Run (), Bit::Vector (7.3), CPAN::DistnameInfo (0.12), CPAN::Meta (2.132830), CPAN::Meta::Check (0.008), CPAN::Meta::Requirements (2.125), CPAN::Meta::YAML (0.010), Cache::Cache (1.06), Carp::Clan (6.04), Class::Data::Inheritable (0.08), Class::Load (0.20), Clone (0.35), Convert::Binary::C (0.76), Crypt::RC4 (2.02), DBD::SQLite (1.40), Data (), Data::OptList (0.108), Devel::GlobalDestruction (0.12), Devel::GlobalDestruction::XS (0.01), Devel::StackTrace (1.30), Digest::Perl::MD5 (1.8), Dist::CheckConflicts (0.09), Eval::Closure (0.11), Exception::Class (1.37), File::Slurp (9999.19), File::Which (1.09), Graph (0.96), IO::All (0.51), IO::Prompt (0.997002), IPC::Run3 (0.046), IPC::System::Simple (1.25), Inline (0.53), JSON::PP (2.27203), List::MoreUtils (0.33), MRO::Compat (0.12), Math::Derivative (undef), Math::Random (0.71), Math::Spline (0.02), Module::Implementation (0.07), Module::Metadata (1.000019), Module::Runtime (0.013), Moose (2.1005), OLE::Storage_Lite (0.19), PPM (undef), Package::DeprecationManager (0.13), Package::Stash (0.36), Package::Stash::XS (0.28), Params::Util (1.07), Perl (5.10.1), Perl::OSType (1.007), Perl::Unsafe::Signals (0.02), PerlIO::gzip (0.18), PostScript::Metrics (0.06), Probe::Perl (0.03), Proc::ProcessTable (0.50), SVG (2.59), SVG::Graph (0.02), Set::Scalar (1.26), Spreadsheet::ParseExcel (0.59), Spreadsheet::WriteExcel (2.40), Statistics::Descriptive (3.0605), Sub::Exporter (0.987), Sub::Exporter::Progressive (0.001011), Sub::Install (0.927), Sub::Name (0.05), Sub::Uplevel (0.24), Sys::SigAction (0.21), Term::ReadKey (2.31), Test::CheckDeps (0.010), Test::Deep (0.110), Test::Differences (0.61), Test::Exception (0.32), Test::Fatal (0.013), Test::Most (0.31), Test::NoWarnings (1.04), Test::Requires (0.06), Test::Script (1.07), Test::Warn (0.24), Text::Diff (1.41), Tree::DAG_Node (1.18), Try::Tiny (0.18), Want (0.22), XML::DOM::XPath (0.14), XML::Filter::BufferText (1.01), XML::SAX::Writer (0.53), XML::Twig (3.44), XML::Writer (0.623), XML::XPathEngine (0.14), YAML (0.81), YAML::Syck (1.25), bin::pairfq (), forks (0.34) |- |} =Licence= Differs according to used Perl module. =Use= Initialize environment with command: module add bioperl-1.6.1 You can also install your own Perl module, see [[How to install application#Perl libraries|SW installation]] for details. To list all available perl modules you can use script <tt>perl_installed_modules.pl</tt>. Usage: module add bioperl-1.6.1 perl_installed_modules.pl =Documentation= Perl modules documentation is available through [http://www.cpan.org/ CPAN]. =Program manager= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= URL: http://www.cpan.org/ [[Category:Applications]] [[Category:English]] 2pvz95lobt2u2bzn89ird1wut3j9diy 0 296 5144 4765 2016-05-13T09:14:31Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Sparx|Česká verze--> = Description = SPARX, a new environment for Cryo-EM image processing. = Availability = Daily built 01/10/05. Including MPI support. = Licence = BSD/GNU licence. = Use = module add sparx sparx = Documentation = http://sparx-em.org/sparxwiki/ = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://sparx-em.org/sparxwiki/ [[Category:Applications]] [[Category:English]] dfudrandl7mvso8p4llpna1qexksd5u 0 225 5146 4808 2016-05-13T09:14:34Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Survival_Kit|Česká verze--> ===Description=== '''''Survival Kit''''' is primarily intended to fill a gap in the software available to animal breeders who generally tend to use extremely large data sets and want to estimate random effects. Methods of survival analysis have primarily been developed in the area of clinical biometrics where data sets and number of levels of effects are usually smaller. Although developed by animal breeders for animal breeders, the programs of the Survival Kit could be interesting for people from other areas encountering similar problems of large models and random effects. To make the Survival Kit user-friendly, commands used in the parameter files mimic the SAS command language. The models supported by the Survival Kit belong to the group of univariate proportional hazards models with a single response time. Fixed and random covariates (discrete and continuous) as well as strata may be included in the analysis. Both types of covariates may be time-dependent. Features include likelihood ratio tests for the full model and any submodel, prediction of (e.g.) median survival times for any given combination of covariates and various bells and whistles for complex models and large data sets. With version 6 it is possible to estimate variances of two random effects at the same time using a derivative free algorithm. The Survival Kit consists of three main tools and a set of support programs: '''Main tools:''' * ''prepare.exe'' -- for data preparation * ''cox.exe'' -- for analysis under the Cox model * ''weibull.exe'' -- for analysis under a model with Weibull baseline hazard function '''Support programs:''' * ''digamma.exe'' -- program to transform the value of gamma into trigamma * ''simul.exe'' -- database simulation for the Survival Kit * ''SKonstructor.exe'' -- Java application for parameter file creation * ''sortSKit'' -- R function for sorting of the recoded data file ===Availability=== Survival Kit is available in the version '''6.06'''. ===Use=== * load the module into the environment <source lang="bash"> $ module add survivalkit-6.06 </source> * further, proceed based on the program you intend to use: ====1. Main tools==== '''<code>prepare.exe</code> tool:''' * prepare the program input file (named <code>prepare.txt</code>) into the working directory * run the program <code>prepare.exe</code> in the working directory (the input file will be loaded automatically) <source lang="bash"> $ cd $WORKING_DIRECTORY $ prepare.exe </source> '''<code>cox.exe</code> tool:''' * in the working directory, prepare the output file of the <code>prepare.exe</code> tool (named <code>varlist.txt</code>) and the COX program input file (the description of the statistical model used -- named <code>cox.txt</code>) * run the <code>cox.exe</code> in the working directory ** if the files are named differently, pass their names as the input parameters to COX tool <source lang="bash"> $ cd $WORKING_DIRECTORY $ cox.exe # alternatively: $ cox.exe varlist.txt cox.txt </source> '''<code>weibull.exe</code> tool:''' * in the working directory, prepare the output file of the <code>prepare.exe</code> tool (named <code>varlist.txt</code>) and the WEIBULL program input file (the description of the statistical model used -- named <code>weibull.txt</code>) * run the <code>weibull.exe</code> in the working directory ** if the files are named differently, pass their names as the input parameters to WEIBULL tool <source lang="bash"> $ cd $WORKING_DIRECTORY $ weibull.exe # alternatively: $ weibull.exe varlist.txt weibull.txt </source> ====2. Support programs==== '''<code>digamma.exe</code> program:''' * ''method 1)'' once the program starts, write the value intended to convert into the terminal: <source lang="bash"> $ digamma.exe GAMMA ? 25.1 Digamma(gamma)= 3.2028152753594581 E(loggamma variable)= -2.00525707782803764E-002 Trigamma(gamma)= 4.06448119882078934E-002 </source> * method 2) pass the value intended to convert to the program input: <source lang="bash"> $ echo 25.1 | digamma.exe GAMMA ? Digamma(gamma)= 3.2028152753594581 E(loggamma variable)= -2.00525707782803764E-002 Trigamma(gamma)= 4.06448119882078934E-002 </source> '''<code>simul.exe</code> program:''' * write the input data into the terminal, answering the program questions: <source lang="bash"> $ simul.exe ... </source> '''<code>simul</code> program for ''R'' tool:''' * if you want to generate the simulation database within the ''R'' program, run the ''R'', load the library provided by Survival Kit, and generate the database as intended (passing the proper input parameters): <source lang="bash"> $ module add R # load the R application module $ R # run the R program (the other commands are passed into the R environment) ... (intro text) ... > source("/software/survivalkit-6.06/Survival_Kit_support_programs/R_extensions/simul_for_R/simul.r") > simul(seed=..., n=..., rho=..., tmed=..., tcens=..., nfix=..., nlevfix=c(...,...,...), fixmin=c(...,...,...), fixmax=c(...,...,...), nsire=..., variance=..., nss=..., wdNew="...", wdSetBack=...) > q() Save workspace image? [y/n/c]: n </source> '''<code>SKonstructor.exe</code> program:''' * since the ''SKonstructor.exe'' is a Java-based application, log into a frontend having an X display tunnelled (e.g., <code>ssh -X <frontend></code> from a Linux box) ** since ''SKonstructor.exe'' is a low-demanding application, you can run it directly from a frontend * run the tool <source lang="bash"> $ SKonstructor.exe </source> '''<code>sortSKit</code> program for ''R'':''' * if you want to sort a data file (e.g., for its further use by COX program), run the ''R'' program, load the library provided by Survival Kit, and sort the file as intended (passing the proper input parameters): <source lang="bash"> $ module add R # load the R application module $ R # run the R program (the other commands are passed into the R environment) ... (intro text) ... > source("/software/survivalkit-6.06/Survival_Kit_support_programs/R_extensions/sortSKit/sortSKit.r") > sortSKit("input_file.dat", "output_file.dat", mode="cox") # or another parameters -- see documentation > q() Save workspace image? [y/n/c]: n </source> ===Documentation=== ===='''Survival Kit''' documentation==== is available * locally in the <code>/software/survivalkit-6.06/Survival_Kit_v6_manual.pdf</code> file * ''use case examples'' are available: ** locally in the <code>/software/survivalkit-6.06/Survival_Kit_v6_examples.pdf</code> *** source files for examples are available in the <code>/software/survivalkit-6.06/Survival_Kit_examples</code> directory ** remotely on http://en.wikiversity.org/wiki/Survival_Kit ===='''<code>simul</code> for R''' documentation==== is available * locally in the <code>/software/survivalkit-6.06/Survival_Kit_support_programs/R_extensions/simul_for_R/simul_for_R_manual.pdf</code> file ===='''<code>sortSKit</code> for R''' documentation==== is available * locally in the <code>/software/survivalkit-6.06/Survival_Kit_support_programs/R_extensions/sortSKit/sortSKit_manual.pdf</code> file * remotely on http://upload.wikimedia.org/wikiversity/en/9/93/SortSKit_manual.pdf ===License=== The program is free of charge for research purposes, but the use should be '''acknowledged in publications "''(Ducrocq et al., 2010)''"'''. ===Supported platforms=== Linux ===Program administrator=== Ludmila Zavadilová, Tomáš Rebok [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== * URL: http://www.nas.boku.ac.at/1897.html * URL: http://en.wikiversity.org/wiki/Survival_Kit [[Kategorie:Applications]] pc2r0lg6ojvpnatqi31hk7e3f8t7btj 0 221 5148 4749 2016-05-13T09:14:35Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--INTEL_CDK|Česká verze--> =Description= The '''Intel development kit''' is a set of tools for development of parallel (supports OpenMP and MPI parallel models) as well as serial programs programmed in C, C++, FORTRAN 77, Fortran 95 and High Performance Fortran (HPF). In addition to compilers and support libraries, the kit includes a debugger and a profiler with support for parallel programs, optimized mathematical libraries BLAS and LAPACK and ScaLAPACK parallel library. Included tools: * ''Intel® C/C++ Compiler'' -- C/C++ compilers * ''Intel® Fortran Compiler'' -- Fortran compiler * ''Intel® Debugger (IDB)'' -- a tool for debugging programs (including parallel/distributed ones) written in C, C++, and Fortran * ''Intel® Cilk™ Plus'' -- an extension of C/C++ compilers that simplifies an implemention of parallel applications * ''Intel® MKL (Math Kernel Libraries)'' -- a computing math library of highly optimized, extensively threaded math routines for applications that require maximum performance (core math functions include BLAS, LAPACK, ScaLAPACK1, sparse solvers, fast Fourier transforms, vector math, and more) * ''Intel® IPP (Integrated Performance Primitives)'' -- an extensive library of multicore-ready, highly optimized software functions for multimedia, data processing, and communications applications * ''Intel® TBB (Threading Building Blocks)'' -- runtime library that simplifies the writing of multithreaded applications in the C++ environment * ''Intel® Trace Analyzer and Collector'' -- graphical tool for understanding MPI application behavior, quickly finding bottlenecks, improving correctness, and achieving high performance for parallel cluster applications based on Intel architecture =Availability= {{Warning | The purchased licenses permit '''just an academic use''' of the program!}} '''Intel CDK version 15.0 (module <code>intelcdk-15</code>) -- newly called ''Intel Parallel Studio XE 2015 Cluster Edition'':''' * ''Intel® C/C++/Fortran Compiler'', ''Intel® Debugger'', ''Intel® Cilk™ Plus'', ''Intel® MKL'', ''Intel® IPP'', ''Intel® TBB'' -- '''2x permanent network-floating license''' * ''Intel® Fortran Compiler'' -- '''2x permanent network-floating license''' '''Intel MPI verze 5.0.1 (module <code>intelmpi-5.0.1</code>) * ''Intel® Trace Analyzer and Collector'' '''Intel CDK version 14.0 (module <code>intelcdk-14</code>):''' * ''Intel® C/C++/Fortran Compiler'', ''Intel® Debugger'', ''Intel® Cilk™ Plus'', ''Intel® MKL'', ''Intel® IPP'', ''Intel® TBB'' -- '''2x permanent network-floating license''' * ''Intel® Fortran Compiler'' -- '''2x permanent network-floating license''' '''Intel C/C++ Composer version 13.1 (module <code>intelcdk-13</code>):''' * ''Intel® C/C++ Compiler'', ''Intel® MKL'', ''Intel® TBB'' -- '''4x permanent network-floating license''' '''Intel CDK version 12.1 (module <code>intelcdk-12</code>):''' * ''Intel® C/C++ Compiler'', ''Intel® Debugger'', ''Intel® Cilk™ Plus'', ''Intel® MKL'', ''Intel® IPP'', ''Intel® TBB'' -- '''4x permanent network-floating license''' * ''Intel® Fortran Compiler'' -- '''2x permanent network-floating license''' '''Intel CDK version 11.0 (module <code>intelcdk-11</code> and <code>intelcomp-11</code>):''' * ''Intel® C/C++ Compiler'', ''Intel® Fortran Compiler'', ''Intel® Debugger'', ''Intel® Cilk™ Plus'', ''Intel® MKL'', ''Intel® IPP'', ''Intel® TBB'' -- '''2x permanent network-floating license''' '''Intel CDK version 9.0 (module <code>intelcdk-9</code>):''' * ''Intel® C/C++ Compiler'', ''Intel® Fortran Compiler'', ''Intel® Debugger'', ''Intel® MKL'', -- '''unlimited permanent network-floating license''' '''Intel CDK version 8.1 (module <code>intelcdk-8</code>):''' * ''Intel® C/C++ Compiler'', ''Intel® Fortran Compiler'', ''Intel® Debugger'', ''Intel® MKL'', -- '''2x permanent network-floating license''' =Usage= Initialize an environment and start the required program (icc, icpc, ifort, idb, ...) in accordance with the documentation. <source lang="bash"> module add intelcdk-15 # Intel CDK version 15.0 module add intelcdk-14 # Intel CDK version 14.0 module add intelcomp-13 # Intel Composer version 13.1 module add intelcdk-12 # Intel CDK version 12.1 module add intelcdk-11 # Intel CDK version 11.0 module add intelcomp-11 # Intel Composer version 11.0 module add intelcdk-9 # Intel CDK version 9.0 module add intelcdk-8 # Intel CDK version 8.1 </source> === Linking MKL libraries === [[Soubor:Math_libraries_dependency.png|right|300px]] Simple way, how to get needed linking parameters provides '''[https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor Intel MKL link advisor]'''. '''Note:''' Intel FFTW libraries are [https://software.intel.com/en-us/articles/fft-interfaces-in-intelr-mkl the part of MKL] since version 10. From that version it is sufficient to link only MKL and not bother with other wrappers. ====Linking examples==== These examples assumes using of <tt>intelcdk-15</tt> module. All libraries are accessible at <tt>$MKLROOT/lib/intel64</tt>, differences between LP and ILP versions are described in the article [https://software.intel.com/en-us/node/528682 Using the ILP64 Interface vs. LP64 Interface] on Intel pages. Following parameters can be variously combined. * Using OpenMP: export LDFLAGS="-L$MKLROOT/lib/intel64 -openmp" * Using BLAS, LAPACK: export LDFLAGS="-L$MKLROOT/lib/intel64 -lmkl_blas95_lp64 a -lmkl_lapack95_lp64" * Using SCALAPACK (LAPACK with MPI support), mandatory addition of <tt>openmpi-1.8.2-intel</tt> module: export LDFLAGS="-L$MKLROOT/lib/intel64 -lmkl_blacs_openmpi_lp64 -lmkl_scalapack_lp64" ==== ld: cannot find -lompstub ==== From certain version of Intel compiler is used the library <tt>iomp5</tt> instead of <tt>ompstub</tt>. So with intelcdk-15 use <tt>-liomp5</tt> instead of <tt>-lompstub</tt>. === Programs and environment variables === Quick overview of the most used programs in module to ease you the searching in man pages. {|class="tabulka" |- !Program !Description !Corresponding environment variables !Notes |- | icc | preprocessor, compiler, assembler and linker for C | CC="icc", CPP="icc -E" | |- | icpc | preprocessor, compiler, assembler and linker for C++ | CXX="icpc", CXXCPP="icpc -E" | |- | ifort | preprocessor, compiler, assembler and linker for Fortran 77 | F77="ifort -f77rtl" | default behavior of ifort is like Fortran 90 compiler |- | ifort | preprocessor, compiler, assembler and linker for Fortran 90/95 | F90="ifort", FC="ifort" | default |- |} === GCC and ICC interaction === Program parts can be compiled with different compilers and then link them together. Nice description is [https://software.intel.com/sites/products/documentation/doclib/iss/2013/compiler/cpp-lin/GUID-0E85F327-A19E-410C-8203-0A89C8F32635.htm on Intel pages]. =Documentation= Documentation is available on [http://software.intel.com/en-us/intel-software-technical-documentation producer's WWW server] (Support tab) or locally in the relevant directories: * version 15.0: <code>/software/intelcdk-15/composer_xe_2015.0.090/Documentation/</code> * version 14.0: <code>/software/intelcdk-14/composer_xe_2013_sp1.0.080/Documentation/</code> * version 13.1: <code>/software/intelcomp-13/composer_xe_2013.5.192/Documentation/</code> * version 12.1: <code>/software/intelcdk-12/composer_xe_2011_sp1.6.233/Documentation/</code> * version 11.0: <code>/software/ict-3.2.020/<PRODUKT>/11.0.074/Documentation/</code> * version 9.0: <code>/software/intel-9.0/amd64_linux26/doc/</code> * version 8.1: <code>/software/intelcomp-8.0/i386/<PRODUKT>/doc/</code> =License= May be used on MetaCentrum nodes by any MetaCentrum member. =Supported platforms= Linux =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= URL: http://software.intel.com/en-us/articles/intel-sdp-products/ [[Category:Applications]] [[Category:English]] gohy4p1itgn1aszud1pme5rr9ryjvcc 0 203 5152 4791 2016-05-13T09:14:38Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--RepeatMasker|Česká verze--> ===Description=== [http://www.repeatmasker.org RepeatMasker] is a program that screens DNA sequences for interspersed repeats and low complexity DNA sequences. The output of the program is a detailed annotation of the repeats that are present in the query sequence as well as a modified version of the query sequence in which all the annotated repeats have been masked (replaced by Ns by default). On average, almost 50% of a human genomic DNA sequence currently will be masked by the program. Sequence comparisons in our copy of RepeatMasker are performed by RMBlast search engine. ===Version=== RepeatMasker version 3.3.0 is currently installed. ===Submodules=== RepeatMasker uses the following submodules: * [http://tandem.bu.edu/trf/trf.html Tandem Repeats Finder] version 3.2.1 * [http://www.repeatmasker.org/RMBlast.html RMBlast] search engine version 1.2 * [http://www.girinst.org Repbase Update] Database of Repetitive DNA version 20120418 ===License Terms and User Agreements=== To use RepeatMasker and its submodules you '''[http://metavo.metacentrum.cz/en/myaccount/licence.html must agree]''' with the following license terms and agreements: * [http://www.repeatmasker.org RepeatMasker] is licensed under the [http://www.opensource.org/licenses/osl-2.1.php Open Source License v2.1]. * [http://tandem.bu.edu/trf/trf.html Tandem Repeats Finder] is licensed under the following terms: ** The author of this software grants to any individual or organization the right to use and to make an unlimited number of copies of this software. You may not de-compile, disassemble, reverse engineer, or modify the software. This software cannot be sold, incorporated into commercial software or redistributed. The author of this software accepts no responsibility for damages resulting from the use of this software and makes no warranty or representation, either express or implied, including but not limited to, any implied warranty of merchantability or fitness for a particular purpose. This software is provided as is, and the user assumes all risks when using it. ** Please cite: G. Benson, "Tandem repeats finder: a program to analyze DNA sequences" Nucleic Acids Research (1999) Vol. 27, No. 2, pp. 573-580. * [http://www.repeatmasker.org/RMBlast.html RMBlast] is licensed under the [http://www.opensource.org/licenses/osl-2.1.php Open Source License v2.1]. * [http://www.girinst.org Repbase Update] is a Database of Repetitive DNA published by Genetic Information Research Institute. You may use the content of the Database free of charge under the following conditions: ** You agree NOT to make the Repbase Update (or any part thereof, including Repbase Reports, Repeat Maps and other derived materials, modified or not) available to anyone outside your research group. "Make available" includes leaving the data where it may be accessible to outside individuals without your direct knowledge (e.g. on a computer to which people outside your group have login privileges), as well as directly providing it to someone. Refer any requests for the Database to [http://www.girinst.org GIRI]. ** You agree NOT to use the Repbase Update for commercially restricted sequencing and/or proprietary sequence analysis. Commercially restricted sequencing is defined as sequencing for which a company retains patenting or licensing rights regarding the sequence, or the right to restrict or delay dissemination of the sequence; with the sole exception that sequencing is not considered to be commercially restricted if it is federally funded and the investigators adopt the data release policies endorsed at the Wellcome Trust-sponsored Bermuda meeting, i.e. immediate release of data as it is generated). ** If you are doing commercially restricted sequencing or other proprietary activities involving any portion of Repbase Update you must commercial register at [http://www.girinst.org/accountservices/register.php GIRI]. ** You agree to properly cite the Database and its specific, original contributions if directly related to your work ([http://www.girinst.org/repbase/update/howtocite.html details]). ** You certify that you are authorized to accept this agreement on behalf of your institution. ** All members of your group with access to the Database agree to the same conditions. ===Documentation=== See online documentation [http://www.repeatmasker.org/webrepeatmaskerhelp.html here] or type: <source lang="bash"> $ module add repeatmasker $ repeatmasker --help </source> ===Usage=== ====Basic usage==== Load SW module repeatmasker, then run RepeatMasker (e.g. with sample input file my_repeatmasker_sample.fasta): <source lang="bash"> $ module add repeatmasker $ repeatmasker my_repeatmasker_sample.fasta </source> In the following text we explain how to run an interactive/batch RepeatMasker job. ====Interactive job==== Interactive job can be run as follows: <source lang="bash"> skirit$ qsub -I -q short -l nodes=1:ppn=1,mem=1000mb </source> (-I stands for interactive, -q short means the job is expected to last less than 2 hours, -l nodes=1:ppn=1,mem=1000mb means one node with one CPU and 1000 MB memory allocation is expected) You are then redirected to a concrete machine where you can run RepeatMasker with my_repeatmasker_sample.fasta input file as follows (and then exit from the machine): <source lang="bash"> $ module add repeatmasker $ repeatmasker my_repeatmasker_sample.fasta ... $ exit </source> Please note that interactive regime does not bring any significant speed-up comparing to running RepeatMasker locally on your machine unless parallelism is used. Interactive regime may be used to test the execution of your job (strongly recommended) and on success you are invited to switch to running it in a batch (see below). ====Batch job==== This is the prefered way of running jobs. Put your input files (e.g. my_repeatmasker_sample.fasta) to a subdirectory of your home directory (which is shared on all machines). Create the shell script my_repeatmasker_script.sh with the following contents: <source lang="bash"> #!/bin/bash # add RepeatMasker module module add repeatmasker # go to the directory where the sample file my_repeatmasker_sample.fasta is located cd /storage/home/`whoami`/subdirectory_with_the_sample_file # run RepeatMasker repeatmasker my_repeatmasker_sample.fasta </source> Submit this shell script by something like: <source lang="bash"> qsub -q short -l nodes=1:ppn=1,mem=1000mb my_repeatmasker_script.sh </source> If you expect the job to last more than two hours use -q normal (up to 24 hours) or -q long (up to 30 days). The -l parameters are to be set according to your expectation of the job resource requirements. ===Program administrator=== [mailto:mjaros@ics.muni.cz Michal Jaros] ===Useful links=== *[http://www.repeatmasker.org http://www.repeatmasker.org] *[http://tandem.bu.edu/trf/trf.html http://tandem.bu.edu/trf/trf.html] *[http://www.girinst.org http://www.girinst.org] [[Kategorie:Applications]] o61i6jcp3yapfv3748pb37fv4vg453z 0 394 5158 4754 2016-05-13T09:14:42Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--USPEX|Česky--> =Description= UPSEX (Universal Structure Predictor: Evolutionary Xtallography) is a method developed by the Oganov laboratory. USPEX method/code solves a problem of crystal structure prediction. =Availability= Application is available in module: * Module <code>uspex-9.4.2</code>: Version 9.4.2 =Use= To run the application use following commands <source lang="bash"> $ module add uspex-9.4.2 $ USPEX </source> =Documentation= You can find documentation in /software/uspex/9.4.2/doc/uspex_manual.pdf or online on the web page http://uspex.stonybrook.edu/documentation.html =Licence= Available for free for academic users. =Program administrator= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= URL: http://uspex.stonybrook.edu/uspex.html [[Kategorie:Applications]] [[Kategorie:English]] 2pe9mvl23uteykuockfnbtq5j5o6cv0 0 209 5160 4759 2016-05-13T09:14:44Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--CMAQ|Česká verze--> ===Description=== The '''EPA Community Multiscale Air Quality (CMAQ)''' modeling system is a third-generation air quality model. CMAQ is an active open-source development project of the U.S. EPA Atmospheric Science Modeling Division, which is designed for applications ranging from regulatory and policy analysis to understanding the complex interactions of atmospheric chemistry and physics. It is a three-dimensional Eulerian (i.e., gridded) atmospheric chemistry and transport modeling system that simulates ozone, particulate matter (PM), toxic airborne pollutants, visibility, and acidic and nutrient pollutant species throughout the troposphere. Designed as a “one-atmosphere” model, CMAQ can address the complex couplings among several air quality issues simultaneously across spatial scales ranging from local to hemispheric. ===Availability=== * CMAQ version '''5.0.1''' (module <code>cmaq-5.0.1</code>): 64-bit, compiled with MPI support (system MPICH2). * CMAQ version 5.0 (module <code>cmaq-5.0</code>): 64-bit, compiled with MPI support (own MPICH2). ===Use=== '''''Important:''' Because of the method the CMAQ is configured as well as performs computations (these are being defined in the CMAQ working directory), the prepared application module performs a symbolic subtree with symbolic links pointing to relevant files in the working directory. These link should be further adapted (by copying and editing) by users to comply with their needs. '''1. Preparation of the working directory:''' * ''(necessary just once)'' * log in to a frontend and change the working directory to a directory, which you intend to use for CMAQ computations * load the CMAQ module <source lang="bash"> $ module add cmaq-5.0.1 </source> :* the module creates a symbolic subtree (within the <code>CMAQv5.0.1</code> directory) with symbolic links pointing to relevant files in the working directory :* to adapt the computations to your need, besides other do: :** copy (or link) your data to compute into the <code>CMAQv5.0.1/data</code> subdirectory :** by copying and subsequent editing adapt the startup scripts to your needs (e.g., the <code>run.cctm</code> script) '''2.1. Starting the computation -- ''Interactive mode:''''' * ask the scheduler for an interactive job having with a desired number of nodes and processors (<code>ppn</code> attribute) reserved <source lang="bash"> $ qsub -I -l nodes=X:ppn=Y -l mem=Zg </source> :''Note: Do not forget to apppropriately set the amount of requested memory (<code>mem</code> attribute) and/or another job requirements.'' :'''''Attention:''' The CMAQ v.5.0 is able to use just the processors on a single computing node (because of the use of its own MPICH2 library).'' * change the working directory to a directory you intended to use for CMAQ (see section 1) and based on your needs adapt the startup scripts to your needs ** besides other, appropriately set the variables '''<code>NPCOL_NPROW</code>''' and '''<code>NPROC</code>''' within the <code>scripts/run.cctm</code> script * start the computation (e.g., in the following order: ICON, BCON, JPROC, CCTM) '''2.1. Starting the computation -- ''Batch mode:''''' * prepare the job description script -- for example, in the following form <source lang="bash"> #!/bin/bash ## set the location, where we have the CMAQ prepared (see section 1) export CMAQDIR=/storage/brno2/home/$USER/CMAQv5.0.1/ ## choose, whether you want to run CMAQ from /scratch or from the location above (CMAQ directory) ## 1. running from /scratch cp -r $CMAQDIR $SCRATCHDIR export M3HOME="$SCRATCHDIR/CMAQv5.0.1/" ## 2. running from the working directory # export M3HOME=$CMAQDIR echo '======================== Running the ICON ========================' ( cd $M3HOME/scripts/icon && ./run.icon ) echo '======================== Running the BCON ========================' ( cd $M3HOME/scripts/bcon && ./run.bcon ) echo '======================== Running the JPROC ========================' ( cd $M3HOME/scripts/jproc && ./run.jproc ) echo '======================== Running the CCTM ========================' ( cd $M3HOME/scripts/cctm && ./run.cctm ) ## if we used the /scratch, copy out the relevant output data and clean the $SCRATCHDIR # cp ... /storage/... rm -rf $SCRATCHDIR/* </source> * pass the job description file to the scheduler together with (at least) the requested number of nodes and processors and the requested amount of memory <source lang="bash"> $ qsub -l nodes=X:ppn=Y -l mem=Zg mydecriptionscript.sh </source> :'''''Attention:''' The CMAQ v.5.0 is able to use just the processors on a single computing node (because of the use of its own MPICH2 library).'' ===Documentation=== * http://www.cmaq-model.org/cmaqwiki/index.php?title=CMAQ_version_5.0_%28February_2010_release%29_OGD * http://www.epa.gov/AMD/CMAQ/CMAQscienceDoc.html ===License=== open-source ===Supported platforms=== amd64 ===Program administrator=== Tom Rebok [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.cmaq-model.org/ [[Kategorie:Applications]] 0tjhlqardxsd22t5xccg2bp8ctm5yp3 0 334 5162 5108 2016-05-13T09:14:46Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Blender|Česká verze--> = Description = Blender is a free and open-source 3D computer graphics software product used for creating animated films, visual effects, art, 3D printed models, interactive 3D applications and video games. Blender's features include 3D modeling, UV unwrapping, texturing, rigging and skinning, fluid and smoke simulation, particle simulation, soft body simulation, sculpting, animating, match moving, camera tracking, rendering, video editing and compositing. Alongside the modelling features it also has an integrated game engine. = Availability = Version 2.70a. Freely available for every user. = Licence = [http://www.gnu.org/copyleft/gpl.html GENERAL PUBLIC LICENSE] = Use = Initialize module with command: module add blender-2.70a {{Application with GUI}} ===Running Blender in than one node=== If you want to run Blender across several nodes, please consult [http://wiki.blender.org/index.php/Doc:2.6/Manual/Render/Performance/Netrender this manual]. = Documentation = No documentation available. = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://www.blender.org/ [[Category:Applications]] [[Category:English]] tu3dzanfcd61dyd11b5mdtdp4foejl3 0 328 5164 4750 2016-05-13T09:14:49Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--FLASH|Česká verze--> = Description = FLASH (Fast Length Adjustment of SHort reads) is a very fast and accurate software tool to merge paired-end reads from next-generation sequencing experiments. FLASH is designed to merge pairs of reads when the original DNA fragments are shorter than twice the length of reads. The resulting longer reads can significantly improve genome assemblies. They can also improve transcriptome assembly when FLASH is used to merge RNA-seq data. = Availability = Version 1.2.9. Freely available to users. Modules: flash-1.2.9 = Licence = [http://www.gnu.org/copyleft/gpl.html GPLv3] = Use = Initialize environment with command: module add flash-1.2.9 = Documentation = Documentation is in file <tt>/software/flash-1.2.9/doc/README</tt>, quite explicit is also a command flash --help = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://ccb.jhu.edu/software/FLASH/ [[Category:Applications]] [[Category:English]] mysu2211dyuqpbacu27n7d0zwuwnu0m 0 329 5165 4781 2016-05-13T09:14:50Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Trimmomatic|Česká verze--> = Description = Trimmomatic performs a variety of useful trimming tasks for illumina paired-end and single ended data.The selection of trimming steps and their associated parameters are supplied on the command line. = Availability = Version 0.32. Freely available to users. Modules: trimmomatic-0.32 = Licence = [http://www.gnu.org/copyleft/gpl.html GPLv3] = Use = Initialize environment with command: module add trimmomatic-0.32 There is defined an environment variable <tt>TRIMMOMATIC_BIN</tt> which points to the program jar file. Example of running: java -jar /software/trimmomatic-0.32/dist/jar/trimmomatic-0.32.jar If you do not need to set any special parameters to java, you can use the available alias to above command called <tt>trimmomatic</tt> . = Documentation = Documentation is in file <tt>/software/trimmomatic-0.32/doc/TrimmomaticManual_V0.32.pdf</tt> = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://www.usadellab.org/cms/?page=trimmomatic [[Category:Applications]] [[Category:English]] kq0xpl8r4z2pb5f0loeabc5s161to4n 0 360 5166 4731 2016-05-13T09:14:51Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--CPLEX Optimizer|Česká verze--> =Description= High-performance mathematical programming solver for linear programming, mixed integer programming, and quadratic programming =Availability= Version 12.5 and 12.6. Freely available to all academic users. '''You have to accept a [https://perun.metacentrum.cz/metavoint/?r=%2fperun-registrar-fed%2f%3fvo%3dmeta%26locale%3den%26group%3dlic_IBM-academic licence agreement] to use this program.''' Available modules: cplex-12.6.1 cplex-12.6.0 cplex-12.5.1 =Licence= [https://www-304.ibm.com/ibm/university/academic/pub/page/mem_program_agreements IBM Academic Initiative License] =Use= You can load the module via the command module add cplex-12.6.1 ===Parallel run=== CPLEX utilizes OpenMP system (parallelization through many processor cores in one machine). You can restrict the number of cores by set up of system variable <tt>OMP_NUM_THREADS</tt>, for example: export OMP_NUM_THREADS=$PBS_NUM_PPN which uses the number of processors granted by planning system. You can also restrict the number of cores directly in the program by set up a proper directive in .mod file: execute{ cplex.threads = XXX; } =Documentation= Documentation is available in html starting in file <tt>/software/cplex-126/doc/html/en-US/documentation.html</tt>. You can find a tutorial in <tt>/software/cplex-126/doc/tutorial/SPSS_Connector.pdf</tt>. ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www-01.ibm.com/software/commerce/optimization/cplex-optimizer/ [[Category:Applications]] [[Category:English]] fovmdrdh64c5q0udfvzweatvpx8md4s 0 162 5167 4785 2016-05-13T09:14:52Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--GARLI|Česká verze--> ===Description=== GARLI (Genetic Algorithm for Rapid Likelihood Inference) is a program that performs phylogenetic inference using the maximum-likelihood criterion. Several sequence types are supported, including nucleotide, amino acid and codon. ===Availability=== Program is available in version 0.96 and 2.01 ===Use=== Initialize enviroment ("module add garli-0.96") and run "Garli0.96b8". For multi processor job ("module add mpich-p4 garli-0.96") and run "mpiexec Garli0.96b8". For the newest 2.01 use module add garli-2.01 Garli nebo GarliMPI ===Documentation=== Documentation is available on [https://www.nescent.org/wg_garli/Main_Page WWW server]. ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: https://www.nescent.org/wg_garli/Main_Page [[Kategorie:Applications]] d28l8548qi5c8reuw3yw1vvvyvuvay1 0 257 5168 4771 2016-05-13T09:14:52Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Turbomole|Česká verze--> ===Description=== '''TURBOMOLE''' is a powerful general purpose Quantum Chemistry (QC) program package for ab initio Electronic Structure Calculations. Presently, the TURBOMOLE is one of the fastest and most stable codes available for standard quantum chemical applications (Hartree-Fock, DFT, MP2, Coupled Cluster, TDDFT, ...). Unlike many other programs, the main focus in the development of TURBOMOLE has not been to implement all new methods and functionals, but to provide a fast and stable code which is able to treat molecules of industrial relevance at reasonable time and memory requirements. Especially the RI-DFT method often saves a factor 10 in CPU-time compared with many other QM programs. TURBOMOLE runs under LINUX, several UNIX variants, Windows, and MacOS in serial and parallel mode. ===Availability=== Versions 5.6, 5.10, 6.0 and 6.4. Currently there is only a licence for a certain group at JCU in MetaCentrum. The TURBOMOLE is available in the following versions: * '''version 6.5''' (module <code>turbomole-6.5</code>) -- a license purchased by Centre CERIT-SC available '''for all the registered MetaVO users''' * versions 6.0 and 6.4 (modules <code>turbomole-6.0</code> and <code>turbomole-6.4</code>) -- licensed for a JCU group * versions 5.6 and 5.10 (modules <code>turbomole-5.6</code> and <code>turbomole-5.10</code>) -- licensed for NCBR research group * module <code>turbomole</code> will always point to '''the newest available version''' === License === * the program is available '''just for non-commercial use''' * '''''version 6.5 is available for all the users (belonging to an institution, which is entitled to provide academic degrees) registered to MetaVO''''' ** [[Media:Turbomole-END USER LICENSE AGREEMENT.pdf|Turbomole End User License Agreement]] ** before being able to use the program, one is required to express an explicit agreement with the license terms and conditions via the [http://metavo.metacentrum.cz/en/myaccount/licence.html MetaVO portal] ===Use=== Single processor version module add turbomole-6.0 jobex -ri >jobex.out Parallel processor version (setting of variables lead to appending of parallel binaries) export PARA_ARCH=MPI export PARNODES=PROCS module add turbomole-6.0 module add openmpi jobex -ri >jobex.out where instead of PROCS specify number of processors. ===Documentation=== Available [http://www.turbomole.com/ on-line on the producer's webpage] or locally in the program directory (/software/turbomole-6.5/DOC/). ===Program manager=== Pavel Fibich [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.turbomole.com/ [[Kategorie:Applications]] dshl04n3ni5rabm6eadsnq5okj9phhw 0 198 5169 5015 2016-05-13T09:14:53Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Edena|Česká verze--> = Description = Edena is one of very efficient and widely used for de novo assembly of short-lenght reads, which are outcome of new generation sequencing machines. It handles several millions of short reads (even pair-end reads) and create contigs on the basis of OLC (Overalp Layout Consensus) algorithm. Edena was developed by Dr. Hernandez and his colleagues (Hernandez D, François P, Farinelli L, Osterås M, Schrenzel J. Genome Research. 18:802-809, 2008.) Edena is still under constant development. This software is very user-friendly. = Usage = Edena version 3 is available as module in METACentrum (module add edena-3). = Documentation = Documentation is available at downloading any Edena version from official website. Program help is also accessible in program when running without an argument. = Supported platforms = Linux x86_64 = Program administrator = * Jan Ištvánek -- Laboratory of molecular plant genetics, PřF MU (janites at mail.muni.cz, i.jan at centrum.cz) * Tomáš Rebok (rebok at ics.muni.cz) = Homepage = URL: http://www.genomic.ch/edena.php [[Kategorie:Applications]] 10yys780wy6ssd6bhxhhtn30jxt8guy 0 272 5173 4745 2016-05-13T09:14:56Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--ERNE|Česká verze--> = Description = ERNE (Extended Randomized Numerical alignEr) is a short string alignment package whose goal is to provide an all-inclusive set of tools to handle short (NGS-like) reads. = Availability = Version 1.2. Freely available for every user. Available modules: erne-1.2 erne-1.4-gcc erne-1.4-intel The difference between -gcc and -intel version is in used build environment. Intel version should be faster, GCC version is more verified so possibly more stable. = Licence = GNU/GPL v3, see http://gplv3.fsf.org/ for details. = Use = Example of module initialization: module add erne-1.2 After initialization are available program binaries and module openmpi-1.6. {{MPI application}} = Documentation = http://erne.sourceforge.net/manual.php = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://erne.sourceforge.net [[Category:Applications]] [[Category:English]] gebhuw2z0jvp4zxmudfkwz1esr9cb0x 0 308 5176 4760 2016-05-13T09:15:02Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--GSL|Česká verze--> = Description = The GNU Scientific Library (GSL) is a numerical library for C and C++ programmers. The library provides a wide range of mathematical routines such as random number generators, special functions and least-squares fitting. There are over 1000 functions in total with an extensive test suite. = Availability = Version 1.16. Freely available for every user. Available modules: gsl-1.16-gcc gsl-1.16-intel The difference between -gcc and -intel version is in used build environment. Intel version should be faster, GCC version is more verified so possibly more stable. = Licence = GNU/GPL v3, see http://gplv3.fsf.org/ for details. = Use = Example of module initialization: module add gsl-1.16-gcc After initialization are available program binaries and libraries. = Documentation = http://www.gnu.org/software/gsl/manual/html_node/ = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://www.gnu.org/software/gsl/ [[Category:Applications]] [[Category:English]] fk8lfb65s14y14bh68x38yxay6n0e9b 0 268 5179 5150 2016-05-13T09:15:03Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--SoX|Česká verze--> = Description = SoX is a cross-platform (Windows, Linux, MacOS X, etc.) command line utility that can convert various formats of computer audio files in to other formats. It can also apply various effects to these sound files, and, as an added bonus, SoX can play and record audio files on most platforms. = Availability = Version 14.4.1. Zkompilováno s podporou externích kodeků: libmp3lame, FLAC, libogg, libvorbis. Freely available to all users. = Licence = LGPL v2 or GPL v2 = Use = Initialize module with command: module add sox-14.4.1 After initialization are available all required program binaries and system variables for possible compilation of another programs. = Documentation = http://sox.sourceforge.net/Docs/Documentation = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://sox.sourceforge.net/ [[Category:Applications]] [[Category:English]] n0cbdo9d5a1nh65o1zlyzyn28n7fss4 0 201 5180 5069 2016-05-13T09:15:04Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Affymetrix Power Tools|Česká verze--> ===Description=== Affymetrix Power Tools (APT) are a set of cross-platform command line programs that implement algorithms for analyzing and working with Affymetrix GeneChip® arrays. APT programs are intended for "power users" who prefer programs that can be utilized in scripting environments and are sophisticated enough to handle the complexity of extra features and functionality. The vision is that APT provides a platform for developing and deploying new algorithms without waiting for the GUI implementations. (Taken from APT homepage.) ===Availability=== Version 1.14.4 will be soon available as a module. ===Use=== NOT READY YET ===Documentation=== http://media.affymetrix.com/support/developer/powertools/changelog/index.html ===Supported platforms=== amd64_linux26 ===Program administrator=== [mailto:morkovsk@natur.cuni.cz morkovsk@natur.cuni.cz] ===Homepage=== http://www.affymetrix.com/partners_programs/programs/developer/tools/powertools.affx ===Licesnse=== APT is an open-source project licensed under the GNU General Public License (GPL). (Developers who need a non-GPL license may purchase a commercial license from Affymetrix.) ===Parallelized workflow=== <source lang="bash"> # coming later </source> [[Category:English]] [[Kategorie:Applications]] n45qb8vyaav6xlz0lvdp5mzp0yrdq4t 0 156 5181 4902 2016-05-13T09:15:04Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--PC_GAMESS-Firefly|Česká verze--> ===Description=== The PC GAMESS/Firefly is a freely available ab initio and DFT computational chemistry program. PC GAMESS/Firefly shares lots of functionality with the GAMESS(US). All the functionality of October 25, 1999 release of GAMESS (US) is supported. Many additional features added to GAMESS (US) since 1999 are supported as well. PC GAMESS/Firefly extends functionality of the GAMESS(US) program by providing very fast Moller-Plesset second, third and fourth order correlation corrections based on the self-consistent electron pairs (SCEP) theory. ===Availability=== Versions PCGamess 7.10 a PCGAMESS 7.1e ===Licence=== It is necessary to agree with a [http://metavo.metacentrum.cz/en/myaccount/licence.html licence] ===Use=== Load modul ("module ad pcgamess-7.1" for version 7.10) or ("module ad pcgamess" for version 7.1e) and run program. ===Documentation=== Online version: http://classic.chem.msu.su/gran/gamess/index.html ===Supported platforms=== Linux ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://classic.chem.msu.su/gran/gamess/index.html [[Kategorie:Applications]] r74rlmoc6vpu1np9affijp69okaxg3m 0 338 5184 4796 2016-05-13T09:15:06Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--PASHA|Česká verze--> = Description = PASHA is a parallel short read assembler for large genomes using de Bruijn graphs. Taking advantage of both shared-memory multi-core CPUs and distributed-memory compute clusters, PASHA has demonstrated its potential to perform high-quality de-novo assembly of large genomes in reasonable time with modest computing resources. = Availability = Version 1.0.10. Freely available to users. Modules: pasha-1.0.10 = Licence = [http://creativecommons.org/licenses/by-sa/4.0/ CC BY-SA 4.0] &ndash; volně k použití = Use = Initialize environment with command: module add pasha-1.0.10 = Documentation = Basic documentation is at http://pasha.sourceforge.net/homepage.htm#installation. = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://pasha.sourceforge.net/homepage.htm [[Category:Applications]] [[Category:English]] s6r18hrbsuavp3iyflwcyafxzkq1ws5 0 275 5187 5120 2016-05-13T09:15:08Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Open3DQSAR|Česká verze--> = Description = Open3DQSAR is an open-source software aimed at high-throughput chemometric analysis of molecular interaction fields. = Availability = Version 2.23. Freely available to users. There are two versions available. The one which uses [http://math-atlas.sourceforge.net/ ATLAS] library and the one using [http://software.intel.com/en-us/intel-mkl Intel MKL]. = Licence = [http://www.gnu.org/licenses/gpl-3.0.html GNU/GPL v3]. = Use = Initialize environment with command: module add open3dqsar-atlas or module add open3dqsar-intel Initialization makes available the binary <tt>open3dqsar</tt>. = Documentation = Dokumentation is available at http://open3dqsar.sourceforge.net/?Description:Usage, or in the program dir: <tt>/software/o3dqsar-2.23/doc</tt>. === Program manager === [mailto:meta@cesnet.cz meta@cesnet.cz] === Homepage === http://open3dqsar.sourceforge.net/ [[Category:Applications]] [[Category:English]] 9r8nn0io0goruu7f2393qd1t93hx3vk 0 238 5189 4789 2016-05-13T09:15:10Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Ray|Česká verze--> ===Description=== Ray is a parallel software that computes de novo genome assemblies with next-generation sequencing data. Ray is written in C++ and can run in parallel on numerous interconnected computers using the message-passing interface (MPI) standard. ===Availability=== Versions 2.3.1 and 2.0.0 ===Use=== module add ray-2.3.1 mpiexec Ray ... see Documentation. ===Documentation=== http://denovoassembler.sourceforge.net/manual.html ===Licence=== GNU General Public licence, verze 3. ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://denovoassembler.sourceforge.net/ [[Kategorie:Applications]] tdxsbxvyg0qmfwrz9mugisq4s38gzua 0 345 5190 5136 2016-05-13T09:15:10Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Lagrange|Česká verze--> = Description = Lagrange is a Python package implementing likelihood models for geographic range evolution on phylogenetic trees, with methods for inferring rates of dispersal and local extinction and ancestral ranges. = Availability = Version 0.20. Freely available for every user. Available modules: lagrange-0.20-gcc = Licence = [http://www.gnu.org/licenses/gpl-2.0.html GNU/GPL v2] = Use = Example of module initialization: module add lagrange-0.20-gcc After initialization are available program binaries and libraries. = Documentation = Documentation is [https://github.com/blackrim/lagrange/blob/master/manual.pdf?raw=true here], you can find a local copy in <tt>/software/lagrange-0.20/gcc/share/doc/manual.pdf</tt>. = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = https://code.google.com/p/lagrange/ [[Category:Applications]] [[Category:English]] 1sv94xe3fcpfspigv5e682u2mfosviz 0 293 5192 5047 2016-05-13T09:15:12Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--RepeatExplorer|Česká verze--> = Description = RepeatExplorer is a computational pipeline for discovery and characterization of repetitive sequences in eukaryotic genomes. The pipeline uses high-throughput genome sequencing data as an input and performs graph-based clustering analysis of sequence read similarities to identify repetitive elements within analyzed samples. The analysis principles were described in Novak et al. (2010) and examples of its application can be found in a number of published papers (see Appendix). It should be noted that although the repeat identification algorithm generally works for any genome, some parts of the pipeline (e.g. protein domain-based classification of mobile elements) were primarily developed for application to plant genomics. However, there is a possibility to supply a custom repeat database to improve sensitivity in classification of non-plant repeats. = Availability = Version actual at 11.10.2013 = Licence = GPL version 3 = Use = module add repeatexplorer seqclust_cmd.py -h = Documentation = http://repeatexplorer.umbr.cas.cz/static/html/help/manual.html Papers about RE: http://www.biomedcentral.com/1471-2105/11/378 and http://bioinformatics.oxfordjournals.org/content/29/6/792 = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://repeatexplorer.umbr.cas.cz/ [[Category:Applications]] [[Category:English]] fk1zbw1pqn8okdrpc4w0e03kb1nm7hq 0 320 5193 4897 2016-05-13T09:15:13Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Maker|Česká verze--> = Description = MAKER is a portable and easily configurable genome annotation pipeline. Its purpose is to allow smaller eukaryotic and prokaryotic genome projects to independently annotate their genomes and to create genome databases. MAKER identifies repeats, aligns ESTs and proteins to a genome, produces ab-initio gene predictions and automatically synthesizes these data into gene annotations having evidence-based quality values. = Availability = Version 2.28. Freely available to users. Modules: maker-2.28 = Licence = [http://www.gnu.org/copyleft/gpl.html GPLv3] and [http://www.perlfoundation.org/artistic_license_2_0 Artistic licence 2.0] = Use = Initialize environment with command: module add maker-2.28 {{MPI application}} = Documentation = Documentation is at http://www.yandell-lab.org/software/maker_docs.html = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://www.yandell-lab.org/software/maker.html [[Category:Applications]] [[Category:English]] ep3628sy4uja103bs4ewhdlw9newbuc 0 224 5195 4756 2016-05-13T09:15:16Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--VCE|Česká verze--> ===Description=== '''''VCE''''' is a program to estimate covariance matrices in a rather general manner. Two methods are available: * REML (''REstricted Maximum Likelihood'') using analytical gradients, * Gibbs sampling. ===Availability=== VCE is available in the 6.0.2 version. ===Use=== '''Batch use:''' * prepare the job description script -- use a general skeleton supplemented by the following lines: <source lang="bash"> ... # load the VCE module module add vce-6.0 # run the computation vce-6.0.2 my_pfile ... </source> * pass the job description file to the scheduler together with the request for a single processor and the requested amount of memory: <source lang="bash"> $ qsub -l nodes=1:ppn=1 -l mem=XXXg mydescriptionfile.sh </source> '''Interactive use:''' * ask the scheduler for an interactive job having a single node and a single processor reserved: <source lang="bash"> $ qsub -I -l nodes=1:ppn=1 -l mem=XXXg </source> :''Note: Do not forget to apppropriately set the amount of requested memory (<code>mem</code> attribute) and/or another job requirements.'' * load the VCE module into the environment: <source lang="bash"> $ module add vce-6.0 </source> * Run VCE in an interactive mode -- see details in [[#Documentation|documentation]]: <source lang="bash"> $ vce-6.0.2 </source> ===Documentation=== The documentation is available on the [ftp://ftp.tzv.fal.de/pub/latest_vce/doc/ producer's WWW server] or locally in the <code>/software/vce-6.0/doc</code> directory. ===License=== The program is free of charge for research purposes, but the use should be '''acknowledged in publications'''. ===Supported platforms=== Linux ===Program administrator=== Ludmila Zavadilová, Tomáš Rebok [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== URL: http://vce.tzv.fal.de/ [[Kategorie:Applications]] bojb1xkds9q7qzv0udthvxy3yqydbts 0 228 5200 4790 2016-05-13T09:15:20Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Muscle|Česká verze--> ===Description=== MUSCLE is a program for creating multiple alignments of amino acid or nucleotide sequences. A range of options is provided that give you the choice of optimizing accuracy, speed, or some compromise between the two. ===Availability=== Version 3.8.31 ===Use=== module add muscle-3.8.31 muscle see Documentation ===Documentation=== http://www.drive5.com/muscle/docs.htm ===Licence=== Not specified. ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://www.drive5.com/muscle/ [[Kategorie:Applications]] csyg9tsm5zu778cn8uds3a319wh2df8 0 351 5201 4887 2016-05-13T09:15:21Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--SOAPdenovo-trans|Česká verze--> = Description = '''SOAPdenovo-trans''' is a de novo transcriptome assembler basing on the SOAPdenovo framework, adapt to alternative splicing and different expression level among transcripts.The assembler provides a more accurate, complete and faster way to construct the full-length transcript sets. = Availability = * ''version 1.04'' (module <code>soapdenovo-trans-1.04</code>) -- SOAPdenovo-trans version 1.04 = Usage = Load the module (<code>module add soapdenovo-trans-1.04</code>) and use the available tools (<code>SOAPdenovo-127mer, SOAPdenovo-63mer, SOAPdenovo-31mer</code>) in accordance with the program documentation. = Documentation = Documentation is available on official website. Help and instruction manual is also a part of the package. = Supported platforms = Linux x86_64 = Program administrator = * [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = URL: http://soap.genomics.org.cn/SOAPdenovo-Trans.html [[Kategorie:Applications]] izy2i40iany8nfyu3ke5u7x41f5lz12 0 397 5202 4809 2016-05-13T09:15:22Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--TreeMix|Česká verze--> ===Description=== TreeMix is a method for inferring the patterns of population splits and mixtures in the history of a set of populations. In the underlying model, the modern-day populations in a species are related to a common ancestor via a graph of ancestral populations. We use the allele frequencies in the modern populations to infer the structure of this graph. ===Availability=== Program is available in version 1.12 ===Use=== module add treemix-1.12 ===Documentation=== https://bitbucket.org/nygcresearch/treemix/wiki/Home ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== https://bitbucket.org/nygcresearch/treemix/wiki/Home [[Kategorie:Applications]] qebeiz0w4up4odipzhcfgrna2jge9bm 0 346 5203 5058 2016-05-13T09:15:24Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Infernal|Česká verze--> = Description = Infernal is used to search sequence databases for homologs of structural RNA sequences, and to make sequence- and structure-based RNA sequence alignments. Infernal builds a profile from a structurally annotated multiple sequence alignment of an RNA family with a position-specific scoring system for substi- tutions, insertions, and deletions. Positions in the profile that are basepaired in the consensus secondary structure of the alignment are modeled as dependent on one another, allowing Infernal’s scoring system to consider the secondary structure, in addition to the primary sequence, of the family being modeled. Infernal profiles are probabilistic models called “covariance models”, a specialized type of stochastic context-free grammar (SCFG) (Lari and Young, 1990). = Availability = Version 1.1 Freely available for every user. Available modules: infernal-1.1-intel infernal-1.1-pgi infernal-1.1-gcc = Licence = Infernal is licensed and freely distributed under the GNU General Public License version 3 (GPLv3). For a copy of the License, see [http://www.gnu.org/licenses/gpl.html GENERAL PUBLIC LICENSE]. = Use = Initialize module with command: module add infernal-1.1-intel = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://infernal.janelia.org/ [[Category:Applications]] [[Category:English]] ogsocay5hu374db72t491wuwwajvz7f 0 311 5204 4822 2016-05-13T09:15:26Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--INTEGRAL_OSA|Česká verze--> = Description = '''INTEGRAL Off-line Scientific Analysis (OSA)''' is ... = Availability = = Licence = = Use = = Documentation = = Program manager = Filip Münz, Matúš Kocka, Tomáš Rebok ([mailto:meta@cesnet.cz meta@cesnet.cz]) = Homepage = www.isdc.unige.ch/integral/analysis [[Category:Applications]] [[Category:English]] ea6vfkk1tyampreuh24m0tzo88pygl9 0 382 5205 5000 2016-05-13T09:15:28Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--VinaLC|Česká verze--> ===Description=== A very popular PC-based molecular docking program, AutoDock Vina, was modified and parallelized, using an MPI and multithreading hybrid scheme, and potentially can be used in the future on exascale machines, without sacrificing accuracy. The resulting program scales up to more than 15K CPUs with a very low overhead cost. ===Reference=== Xiaohua Zhang, Sergio E. Wong, and Felice C. Lightstone. (2013) Message Passing Interface and Multithreading Hybrid for Parallel Molecular Docking of Large Databases on Petascale High Performance Computing Machines. J. Comput. Chem. 34, 915-927. http://onlinelibrary.wiley.com/doi/10.1002/jcc.23214/abstract ===Availability=== VinaLC version: 1.1.2 ===Use=== module add vinalc-1.1.2 vinalc ===Documentation=== http://mvirdb1.llnl.gov/static_catsid/vina/usersGuide.html ===Licence=== Apache License 2.0 ===Supported platforms=== Linux ===Program administrator=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://mvirdb1.llnl.gov/static_catsid/vina/index.html ====Usage==== Sample of usage: module add vinalc-1.1.2 cd /storage/brno6/home/aqua/vstupy/ mpirun --mca btl ^openib vinalc --recList /storage/brno6/home/vstupy/ligands/receptor_rigid.txt --fleList /storage/brno6/home/aqua/vstupy/ligands/receptor_flex.txt --ligList /storage/brno6/home/aqua/vstupy/ligands/ligands.txt --geoList /storage/brno6/home/aqua/vstupy/ligands/conf-4ey7.txtx > vinalcOutput [[Kategorie:Applications]] [[Category:English]] qvfc1hj0nphaikhbut9kvweprklc3cu 0 368 5209 4833 2016-05-13T09:15:33Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--MP-EST|Česká verze--> =Description= A tool to find microRNAs with high accuracy and no learning at genome scale and from deep sequencing data. =Availability= Version 1.4 and 1.5. Available modules: mpest-1.5 mpest-1.4 =Use= module add mpest-1.4 =Licence= GNU GPL v2 licence =Program manager= [mailto:meta@cesnet.cz meta@cesnet.cz] =Homepage= * https://faculty.franklin.uga.edu/lliu/content/mp-est? * https://code.google.com/p/mp-est/ [[Category:Applications]] [[Category:English]] g2gahrcl55605jyb1k8ewbrliadnf8o 0 216 5211 5025 2016-05-13T09:15:36Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--Bowtie|Česká verze--> ===Description=== Bowtie is an ultrafast, memory-efficient short read aligner. It aligns short DNA sequences (reads) to the human genome at a rate of over 25 million 35-bp reads per hour. Bowtie indexes the genome with a Burrows-Wheeler index to keep its memory footprint small: typically about 2.2 GB for the human genome (2.9 GB for paired-end). Bowtie 2 is an ultrafast and memory-efficient tool for aligning sequencing reads to long reference sequences. It is particularly good at aligning reads of about 50 up to 100s or 1,000s of characters, and particularly good at aligning to relatively long (e.g. mammalian) genomes. Bowtie 2 indexes the genome with an FM Index to keep its memory footprint small: for the human genome, its memory footprint is typically around 3.2 GB. Bowtie 2 supports gapped, local, and paired-end alignment modes. ===Availability=== Bowtie version 0.12.8 and 0.12.9. Bowtie2 version 2.1.0, 2.2.3 and 2.2.4. ===Use=== module add bowtie-0.12.8 bowtie or module add bowtie2-2.1.0 bowtie2 for details see Documentation. ===Documentation=== http://bowtie-bio.sourceforge.net/index.shtml http://bowtie-bio.sourceforge.net/bowtie2/index.shtml ===Licence=== Artistic license (see /software/bowtie-0.12.8/src/COPYING) for bowtie and GNU GPL version 3 for bowtie2. ===Program manager=== [mailto:meta@cesnet.cz meta@cesnet.cz] ===Homepage=== http://bowtie-bio.sourceforge.net/index.shtml http://bowtie-bio.sourceforge.net/bowtie2/index.shtml [[Kategorie:Applications]] tm8qdl9sdl13119mtfcys3k55me5f9o 0 295 5213 4899 2016-05-13T09:15:37Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--SaSSY|Česká verze--> = Description = SaSSY is a short, paired-read assembler designed primarily to assemble data generated using Illumina platforms. SaSSY incorporates a novel contig extension method which does not make heavy use of coverage information. = Availability = Version 0.1.1.3 = Licence = Not specified. = Use = module add sassy-0.1.1.3 Sassy64 newfileset = Documentation = https://github.com/minillinim/SaSSY/blob/master/SaSSY_manual.pdf = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://sassy.mikeimelfort.com/ [[Category:Applications]] [[Category:English]] dckmyuk3ov6923kp9yq43948k45mrfc 0 363 5215 5170 2016-05-13T09:15:39Z Root 1 Nahrazení textu „]])“ textem „-->“ wikitext text/x-wiki <!--YASRA|Česká verze--> = Description = Yet Another Short Read Assembler. = Availability = Version 2.33 = Licence = unknown = Use = module add yasra-2.33 assembler fasta.py genomewelder trim_assembly best_hit genomewalker make_template = Documentation = http://www.bx.psu.edu/miller_lab/ = Program manager = [mailto:meta@cesnet.cz meta@cesnet.cz] = Homepage = http://www.bx.psu.edu/miller_lab/ [[Category:Applications]] [[Category:English]] r1u9yr5u2zg8sganvqfuhocuk9rnmyy 10 404 1948 919 2016-05-03T12:31:56Z Root 1 wikitext text/x-wiki '''Notice:''' This application use or needs GUI &ndash; graphical interface. To use the application in graphical mode see [[Remote desktop]] or [[X-Window]].<br /> [[Category:Applications with GUI]] mletd3n8fm8iaxf51p4varaorj9i3ei 0 531 1899 1898 2016-04-29T17:16:36Z Root 1 /* For Linux users */ wikitext text/x-wiki {{Rámeček-green|text='''Notice:''' Although this manual is still valid, more effective way to use applications with GUI is described on page [[Remote desktop]].}} X-window is a system for sharing the graphical interface of applications (windows) through network. In MetaCentrum you can use it for accessing the graphical interface of certain programs (like [[Matlab application|Matlab]], [[Maple application|Maple]] or [[SNAPP application|SNAPP]]) running in clusters from the user's desktop. =In common= GUI of unix systems are build on X-Window system, which supposes, that a program and his GUI run on different computers in net. Programs in X-Window system display their GUI on so called [http://en.wikipedia.org/wiki/X_Window_System X-server] which is defined by environment variable '''DISPLAY'''. For example: We are sitting in front of computer named ''jenicek'' and start up X-Window program in machine called marenka. If we set variable '''DISPLAY=jenicek:0''' on ''marenka'' and if X-server is running in ''jenicek'', execution of a program will be running in a remote computer, but will be displaying on screen in front us and reacting on keyboard an mouse on our table. There are X-servers for MS Windows, for example [http://sourceforge.net/projects/xming/ Xming], [http://x.cygwin.com/ Cygwin/X], [http://www.starnet.com/products/xwin32/ X-Win32], [http://www.edas.co.uk/connectivity/xwindows/refx.htm ReflectionX] and [http://connectivity.opentext.com/products/exceed-products.aspx Exceed] You can use programs with GUI in MetaCentrum, if this programs supports X-Window. Usually it´s not necessary to set manually the variable DISPLAY, it´s set by ssh run with the option for X11 tunneling. * '''Windows''' &ndash; You should set and fill the correct [http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html PuTTy] [[Media:Putty-x11.png|options]] or use the ''plink'' program from complete PuTTy installation and then use the Linux approach. * '''Unix/Linux''' &ndash; You should add the option <tt>-X</tt> to ssh command. So like: '''ssh -X'''. After the [[frontend]] connection with allowed X11 tunneling you should specify the parameter <tt>-X</tt> for the qsub. So '''qsub -X''' sets the DISPLAY variable on the granted machine. See more info in article [[How to compute/Interactive jobs|Intertive jobs]]. === X-protocol tunneling === The simplest way how to use X-window in MetaCentrum is to submit an '''interactive''' job with X-protocol tunneling enabled by the option '''-X''' for both ssh and qsub: <nowiki>jenicek$ ssh -X skirit.metacentrum.cz skirit$ qsub -X -I -q short -l nodes=1:ppn=1 -l mem=2gb -l matlab=1 konos6$ echo $DISPLAY localhost:50.0 konos6$ module add matlab ; matlab </nowiki> === Direct connection by X-protocol === If you need a direct connection by X-protocol, for example because of reduced latency, you can set the DISPLAY variable on the computing machine to the name of your personal machine: <nowiki>jenicek$ ssh skirit.metacentrum.cz skirit$ qsub -I -q short -l nodes=1:ppn=1 -l mem=2gb -l matlab=1 konos6$ echo $DISPLAY konos6$ export DISPLAY=jenicek.ics.muni.cz:0 konos6$ module add matlab ; matlab </nowiki> On your machine (''jenicek.ics.muni.cz'' in this example) you have to allow incoming connections by jenicek$ xhost + and make sure that your X-server does not have disabled incoming connections. For example, the standard setting on Ubuntu is to start X-server with the options '''-nolisten tcp''' that disables incoming connections. =Step by step= ==For Windows users== [[Soubor:Putty-x11.png|right|thumb|PuTTy setting for X11 tunnel]] # Install [http://sourceforge.net/projects/xming/ Xming]. Run it. # Install [http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html PuTTy]. Run it. # Set the connection parameters in PuTTy according the image in right and save it. # Connect with set up parameters to some [[frontend]]. # Run your [[How to compute/Interactive jobs|interactive job]]. ==For Linux users== # Run the graphical terminal. # Connect to some [[frontend]]. To do it type the command <tt>ssh -X name_of_frontend</tt>. # Run the [[How to compute/Interactive jobs|interactive job]]. # If the error about missing rights to use Xserver occurs, return to point 2 but before connection to the frontend type command <tt>xhost +</tt>. [[Category:Grid services]] [[Category:Applications with GUI]] bjrel7pt081gnf6rifcs5n52scoe963 0 532 1900 1896 2016-04-29T17:28:14Z Root 1 /* Connecting via VNC client */ wikitext text/x-wiki You can use '''VNC Protocol''' to connect to '''remote virtual desktop''' and that way utilize computing power of computational nodes for graphical applications. Compared to [[X-Window]] is this method '''''faster'''''. <!--- and may be used even from a '''''web browser'''''. --> =General information= '''VNC (Virtual Network Computing)''' is a protocol/program which allows remote connection to graphical user interface via computer network. VNC works as a client-server, where the server creates a graphical desktop in a main memory and then communicates with the client, which displays the desktop to user. However, RFB protocol which client and server use for communication, isn't secure and potential intruder might be able to eavesdrop it (including passwords, etc.). That's why was created this howto on '''secure communication''' with MetaCentrum infrastructure. Graphical desktop on server is started by <code>gui</code> script in module ''gui''. Although you'll be able to start it even on [[Frontend|frontend nodes]], it's not a preferred way for computing-heavy tasks. For these it's recommended to start virtual desktop from an [[How to compute/Interactive jobs|interactive job]]. :'''''Note:''' The alternative solution -- tunnelling graphical interface using SSH protocol -- is also supported. For more information see the [[X-Window|X-Window system]] page.'' =Usage= ==General== Connection to frontend node, start of [[How to compute/Interactive jobs|interactive job]] and addition of module ''gui'': <nowiki>jenicek$ ssh skirit.metacentrum.cz skirit$ qsub -I -l walltime=2h -l nodes=1:ppn=1 -l mem=2gb -l matlab=1 took11$ module add gui</nowiki> Apart of model scenarios which are covered later, there are several '''parameters of ''gui'' script you'll find useful''' (see <code>gui --help</code>): * <code>gui '''start''' [-s] [-w] [-g GEOMETRY] [-c COLORS]</code> ... starts a VNC session :* flag <code>-s</code> ... the VNC server will be accessible only via SSH tunnel :* flag <code>-w</code> ... the VNC server will additionally become available via JAVA-enabled web browsers :* flag <code>-g GEOMETRY</code> ... specifies the required screen geometry (e.g. <code>-g 1280x1024</code> - default is 1280x768) :* flag <code>-c COLORS</code> ... specifies the required color depth in bits(e.g. <code>-c 16</code> - default is 24) * <code>gui '''info''' [-p]</code> ... displays information about running VNC sessions '''on the local node''' :* flag <code>-p</code> ... allows to show the VNC session passwords as well * <code>gui '''traverse''' [-p]</code> ... displays information about running VNC sessions '''across all the MetaCentrum nodes''' :* flag <code>-p</code> ... allows to show the VNC session passwords as well :* ''warning: processing of this command may take long time'' * <code>gui '''stop''' DISPLAYID</code> ... stops a VNC session (DISPLAYID may be omitted if single session is running) * <code>gui '''kill''' DISPLAYID</code> ... kills a VNC session (DISPLAYID may be omitted if single session is running) For example: <nowiki> [testvnc@took11 ~]$ gui info -p ***************************************** Your running VNC sessions are: display tunnel machine:port (password) :20 SSL took11.ics.muni.cz:10100 (rxqqOFHK) :21 SSH took11.ics.muni.cz:10102 (IG9Ac1yN) ***************************************** [testvnc@took11 ~]$ gui stop :21 The VNC session running at port 10101 has been successfully stopped... </nowiki> Your virtual desktop will remain active until you stop its interactive job (if run via interactive job), or until you stop it manually (either by command <code>gui stop DISPLAYID</code> or by '''''Start->Logout''''' from graphical interface). Disconnection of VNC client won't affect it in any way. ==Tunneling through SSL (preffered) == '''SSL (Secure Sockets Layer)''' is a protocol which ensures that between transport layer (TCP) and application layer (RFB) is inserted another layer (SSL), which provides authentication and encryption of the communication. In MetaCentrum, you can connect to remote desktop in two ways - using a '''VNC client'''(faster, preferred method) or using a '''web browser'''. ===Connecting via VNC client=== Model example of VNC clients supporting SSL is '''multiplatform [http://sourceforge.net/apps/mediawiki/tigervnc TigerVNC]''' (download '''[http://www.filedropper.com/tigervnc-javaviewer here]'''), which will be assumed to be used in this howto. However, SSL (or more precisely VeNCrypt - the extension of VNC protocol) is supported in several other clients, e.g. most clients included in recent Linux distros (Remmina, KRDC). {{Warning | We '''strongly recommend''' to use '''[http://www.filedropper.com/tigervnc-javaviewer Java version] already linked above''', since other variants (native clients for both Linux and Windows) '''''suffer from frequent freezing of transferred display'''''.}} * after adding a ''gui'' module '''run command <tt>gui start</tt>''', resulting output should look similar to this: <nowiki> took11$ gui start ***************************************** Your VNC session has been started. The connection details are as follows: Host & port : took11.ics.muni.cz:10100 Password : vQFYmf4U Display : :20 ***************************************** </nowiki> * '''run''' on your machine '''TigerVNC client''' * '''copy''' to a ''VNC server:'' field a ''Host & port'' value from former output and choose ''Connect'' * input password from a ''Password'' line <!--{{Warning | If at this stage '''outputs''' Linux version of '''TigerVNC''' client message '''''No matching security types!''''' (happens usually after changing of connection parametrs in a ''Options'' menu), open its configuration file (<code>$HOME/.vnc/default.tigervnc</code>) and ''' add into line "<code>SecurityTypes</code>" value "VeNCrypt"''', like this: <nowiki> ... SecurityTypes=VeNCrypt,TLSVnc,TLSPlain,X509Vnc,X509Plain ...</nowiki> }}--> <!--- ===Connecting via web browser=== For this to work, you need to have installed and activated Java plugin in your web browser. * after adding a ''gui'' module '''run command <tt>gui start --web</tt>''', resulting output should look similar to this: <nowiki> took11$ gui start -w ***************************************** Your VNC session has been started. The connection details are as follows: Host & port : took11.ics.muni.cz:10100 Password : Hf11sYO9 Display : :20 Web URL : http://took11.ics.muni.cz:10101/ ***************************************** </nowiki> * '''enter''' into browser '''web address''' from a ''Web URL'' line * '''confirm''' ''Start of unsigned Java application'' dialog * choose '''''Yes''''' on another query for certificate validity (you have just about 10 seconds for this step; if you run into ''Network error: remote side closed connection'', choose ''Login again'' and repeat above steps as needed) * '''enter password''' from a line ''Password'' '''''Note:''' Even though your browser connects to unsecured HTTP port, after downloading of VNC client applet is all communication secured by SSL.'' --> ==Tunneling through SSH== When '''tunneling through SSH''', the same type of security measures is utilized for VNC protocol data as for your access to remote machines via protocol/command ''ssh''. In the same manner as when connecting to VNC secured by SSL tunnel, you need to start the <tt>gui start</tt> command first, this time with a '''parameter ''--ssh''''' though. * after adding a ''gui'' module '''run command <tt>gui start --ssh</tt>''', resulting output should look similar to this: <nowiki> took11$ gui start --ssh ***************************************** Your VNC session has been started. The connection details are as follows: Remote Host : localhost Port : 10110 Use SSH tun.: yes SSH Server : took11.ics.muni.cz SSH User : testvnc VNC Password: g18LyoAE Display : :23 ***************************************** </nowiki> Then proceed according to your platform - following are howtos for multiplatform '''TightVNC Java Viewer''' client, '''unix shell''' and the most used Windows SSH client - '''PuTTy'''. ===TightVNC=== [[Soubor:tightvnc-javaviewer.png|right|thumb|SSH secured connection using a TightVNC Java Viewer]] The easiest way is probably to use [http://www.tightvnc.com/download/2.7.2/tvnjviewer-2.7.2-bin.zip TightVNC Java Viewer] (needs installed [http://www.oracle.com/technetwork/java/javase/downloads/index.html Java JRE]) - just enter fileds according to <tt>gui start --ssh</tt> output, then first enter your MetaCentrum password (for establishing of a SSH tunnel) and then the temporary password for connecting to your virtual desktop session. ===Putty(Windows)=== [[Soubor:Putty-tunnel.png|right|thumb|Establishing SSH tunel in PuTTy]] * first enter into ''Host name'' field the '''SSH server address''' (see former <tt>gui start --ssh</tt> output, ans if you wanted to connect) * '''select''' ''Connection > SSH > Tunnels'' in a side panel * '''enter value''' of "Port" from former output into ''Source port'' field and set ''Destination'' field to <tt>localhost:PORT</tt> (''PORT'' - again the same number) * '''choose''' ''Add'' and ''Open'' and '''enter password''' to the MetaCentrum infrastructure * use '''any VNC client to connect''' to address ''localhost:PORT'' (here ''localhost:10110'') and '''enter the VNC password''' ===Linux=== * on your local machine run '''ssh -TN -f sshuser@sshserver -L PORT:localhost:PORT''' (''PORT'' - see output of ''gui'') with values from <tt>gui start --ssh</tt> and enter password to the MetaCentrum infrastructure. For output above, it would be: <nowiki> novak$ ssh -TN -f testvnc@took11.ics.muni.cz -L 10110:localhost:10110 testvnc@took11.ics.muni.cz's password: </nowiki> * use '''any VNC client to connect''' to address ''localhost:PORT'' (here ''localhost:10110'') and '''enter the VNC password''' ==Long-standing desktop session== * If you need long-standing remote desktop session with possibility to '''reconnect''', you can use following script (file.sh): <nowiki>#!/bin/bash #PBS -m abe module add gui gui -f start</nowiki> * This script submit as batch job (with specification of memory, time, CPU, ...). E.g. <nowiki>skirit$ qsub -l walltime=24h -l mem=16g file.sh</nowiki> * As soon as the job will be started (you will be informed by e-mail), please log in '''without delay''' to any frontend and type <nowiki>skirit$ module add gui skirit$ gui traverse -p</nowiki> * You can see all running VNC sessions with login information * Using VNC client connect to created session. If you close the window of VNC client, the job won't be stopped. * After end of your work ''' please log out''' (Penguin on the bottom panel -> Logout). Otherwise job end after exhaust reserved time (walltime) [[Kategorie:Grid services]] [[Category:Applications with GUI]] nlq2hy2o41b0h6kfm8tyeiv95wpsnsc 0 499 1381 1380 2016-02-26T19:21:05Z Root 1 naimportována 1 revize wikitext text/x-wiki ([[MetaCloud testbed - popis služby|Czech version]]) All MetaCentrum users can apply for account, no additional verification is required. We will send you an extra password (just for MetaCloud) in reply to your e-mail request. MetaCloud is accessible through web interface at [https://cloud.metacentrum.cz/ MetaCloud] or through [https://meta.cesnet.cz/wiki/MetaCloud_access_through_command_line shell/cmd utilities]. To get further help feel free to e-mail us on [mailto:cloud@metacentrum.cz cloud@metacentrum.cz] or through [https://rt4.cesnet.cz/rt/ RT], queue <tt>cloud</tt>. ===Documentation:=== *[[MetaCloud testbed registration|Registration]] (only through e-mail right now, integration with MetaCentrum in progress) *[[MetaCloud testbed - quick guide|Quick start]] *[[Changing your MetaCloud password|Change password]] *[[MetaCloud testbed - workflow example|Cloud workflow example]] ===Advanced documentation:=== * Access to cloud &ndash; [[Access_to_MetaCloud_via_web_interface | web interface]] &ndash; [[MetaCloud_access_through_command_line|command line]] * Images &ndash; [[Prepared images and templates | prepared]] &ndash; [[Uploading_own_images_to_MetaCloud| own]] &ndash; [[Supported images and conversion | required format]] * Starting and managment of VM &ndash; [[MetaCloud management via web interface | web interface]] &ndash; [[MetaCloud management via command line | command line]] *If you switch from Amazon.com cloud &ndash; [[upload image]] &ndash; [[web interface]] &ndash; [[command line]] &ndash; [[Cumulus_storage|S3 storage]] * Troubleshooting &ndash; [[Known problems in MetaCloud and their solutions|known problems and solutions]] [[Category:English]] [[Category:Clouds]] if22gq469zgw4jj0kyhy3r2ci9td9pr 0 500 1383 1382 2016-02-26T19:21:05Z Root 1 naimportována 1 revize wikitext text/x-wiki ([[FedCloud testbed - popis služby|Czech version]]) This cloud is available for FedCloud members of EGI - European Grid Infrastructure. See [http://www.egi.eu/infrastructure/cloud/] [[Category:English]] [[Category:Clouds]] pinfdyq7kevbxhe2yaewkah8x1cl6ue 0 501 1385 1384 2016-02-26T19:21:05Z Root 1 naimportována 1 revize wikitext text/x-wiki <span style="color:#ff0000">'''THIS PART OF WIKI IS OUTDATED AND UNDER CONSTRUCTION. PLEASE BE PATIENT.'''</span> ([[Přístup_do_MetaCloudu_přes_příkazovou_řádku|Czech version]]) === ONE tools === '''Prerequisite:''' [[MetaCloud_testbed_registration | OpenNebula account]] ONE tools are part of OpenNebula client tools, which are distributed as part of OpenNebula system. It is a set of unix command line utilities, written in Ruby, providing an alternative to Sunstone interface. After first installation and setup are OpenNebula client tools ready and there is no need to install them again in the future. OpenNebula client tools consists of these parts: *ONE tools ONE tools are important for the access to MetaCloud. OCCI and EC2 API are used for uploading images and are described in [[Uploading_own_images_to_MetaCloud|Uploading own images to MetaCloud]]. Despite, it is important to install and setup correctly all three APIs now. '''Prerequisite:''' Unix-type operating system (Windows with Ruby not tested), basic knowledge of such system and editing configuration files *Append the following to ''~/.bashrc'', or ''/home/your_login/.bashrc''. End of the file should look something like this: #select the right host export ONE_HOST=https://cloud.metacentrum.cz #EnvVars for ONE tools export ONE_AUTH=~/.one/one_x509 export ONE_XMLRPC=$ONE_HOST:6443/RPC2 *There are the environment variables of your terminal with Bash interpreter defined in this file. Variables are divided into three sections, based on the interface used for communication with OpenNebula. Please, check if the name of the server is correct (you do not have to pay attention to the ports). *Install '''Ruby''' and '''Rubygems''' *Install the following dependencies and ruby gems sudo apt-get install libexpat1-dev libcurl4-openssl-dev rake libxml2-dev libxslt1-dev gcc libopenssl-ruby make sudo gem install opennebula-cli -v '~> 4.10.2' *Restart the terminal. That way new environment variables are used and ONE tools are ready to use. *It is possible that you received an error if you have Ruby 1.9.3+ installed. "SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed" *Ruby can't find the certificate TERENA-eScience-SSL-CA.pem in /etc/ssl/certs. It can be resolved by following procedure &ndash; Metacentrum certificate installation: sudo -s wget -q https://dist.eugridpma.info/distribution/igtf/current/GPG-KEY-EUGridPMA-RPM-3 gpg --with-fingerprint GPG-KEY-EUGridPMA-RPM-3 pub 1024D/3CDBBC71 2005-07-12 EUGridPMA Distribution Signing Key 3 <info@eugridpma.org> Key fingerprint = D12E 9228 22BE 64D5 0146 188B C32D 99C8 3CDB BC71 apt-key add GPG-KEY-EUGridPMA-RPM-3 echo "deb http://dist.eugridpma.info/distribution/igtf/current igtf accredited" > /etc/apt/sources.list.d/eugridpma.list apt-get update apt-get install ca-policy-igtf-classic ca-policy-igtf-mics ca-policy-igtf-slcs ln -s /etc/grid-security/certificates/9d772be7.0 /etc/ssl/certs/9d772be7.0 ln -s /etc/grid-security/certificates/20ce830e.0 /etc/ssl/certs/20ce830e.0 ln -s /etc/grid-security/certificates/TERENA-eScience-SSL-CA.pem /etc/ssl/certs/TERENA-eScience-SSL-CA.pem *Now the environment is configured for work with MetaCloud via command line. More information about how to work with MetaCloud via command line can be found in the [[MetaCloud management via command line]] section. When using ruby 1.8.7, u may can see following harmless warning, please ignore it. warning: peer certificate won't be verified in this SSL session === Warning === You can't upload any files on the server via ONE tools. You can find out more about this in the [[Uploading_own_images_to_MetaCloud| Uploading own images to MetaCloud]] section. [[Kategorie:English]] [[Kategorie:Clouds]] li7ozdqhqde54u7kpi7wvdzqr49qp4x 0 502 1387 1386 2016-02-26T19:21:05Z Root 1 naimportována 1 revize wikitext text/x-wiki <span style="color:#ff0000">'''THIS PART OF WIKI IS OUTDATED AND UNDER CONSTRUCTION. PLEASE BE PATIENT.'''</span> ([[Předpřipravené obrazy a templaty|Czech version]]) __TOC__ '''Prerequisite''': you have your OpenNebula account ready, ie. access to [https://carach1.ics.muni.cz web interface] or correctly configured and functioning environment with OpenNebula client tools, see tutorial [[MetaCloud access through command line]] To provide a simple start in MetaCloud, we prepared some functional and ready-to-use VM images with appropriate templates for you. *Image is a file, which contains data. In our case its operating system. *Template is a description of the VM (size of the memory, image and network that are used, etc.). Image can be either persistent or non-persistent. *Persistent image preserves all changes and data you make during your work on shutdown. Therefore you can use them next time you instantiate the VM. With this type of image you can't however instantiate it more than once at the time. *Non-persistent image does not save any data nor changes, but you can instantiate many VMs at once. ==Prearranged images== If you open the ''Images'' tab in the left menu in [https://carach1.ics.muni.cz web interface] or use the command line command: oneimage list You can find these images: *''debian6'' &ndash; non-persistent clean Debian 6 image *''scilinux5'' &ndash; non-persistent clean Scientific Linux 5 image *''scilinux6'' &ndash; non-persistent clean Scientific Linux 6 image *''centos5torque'' &ndash; non-persistent image *''centos5openmpi'' &ndash; non-persistent image Persistence of the image is shown in column ''PER'' in command line and in column ''Persistent'' in [https://carach1.ics.muni.cz web interface] ==Prearranged templates== If you open the ''Templates'' tab in the left menu in [https://carach1.ics.muni.cz web interface] or use the command line command: onetemplate list You can find this template: *''debianvm'' &ndash; uses ''debian6'' image, therefore it is not persistent *''scilinux5vm'' &ndash; uses ''scilinux5'' image and it is not persistent *''scilinux6vm'' &ndash; uses ''scilinux6'' image and it is not persistent We can also create templates for other images listed above for you, just write us an [mailto:cloud@metacentrum.cz e-mail]. All these template files contain your ssh key, so you can log into VMs instantiated from these templates without password. [[Category:English]] [[Category:Clouds]] ml7437uafkqo8ub3gti8iwxc8uh71us 0 503 1389 1388 2016-02-26T19:21:06Z Root 1 naimportována 1 revize wikitext text/x-wiki ([[Registrace_do_MetaCloud_testbed|Czech version]]) __TOC__ You can apply for two basic types of accounts. If you only seek to use the basic functionality, most certainly you can apply for ''basic account''. Advanced users might require one of the ''extended accounts''. == How to request an account == << [https://perun.metacentrum.cz/perun-registrar-fed/?vo=meta&targetexisting=https%3A%2F%2Fperun.metacentrum.cz%2Fperun-registrar-fed%2F%3Fvo%3Dmeta%26group%3Dmetacloud&targetnew=https%3A%2F%2Fperun.metacentrum.cz%2Fperun-registrar-fed%2F%3Fvo%3Dmeta%26group%3Dmetacloud&targetextended=https%3A%2F%2Fperun.metacentrum.cz%2Fperun-registrar-fed%2F%3Fvo%3Dmeta%26group%3Dmetacloud MetaCloud registration form] >> ==Basic account== The basic account allows for [[Access to MetaCloud via web interface]] OpenNebula Sunstone. Also, you can upload your own virtual machine images and start/manage/stop virtual machines running from them. Furthermore, you can use all the functionality provided by [http://opennebula.org/documentation:rel3.2:sunstone OpenNebula Sunstone]. In case you want to use the [[Prearranged_images_and_templates|prepared images]], we need you to send us your public ssh key: * to generate your key issue the command ssh-keygen. The options (file name, ...) confirm by pressing ENTER. We recommend that you choose a safe password at the "Enter passphrase (empty for no passphrase)" prompt. ssh-keygen * In your home folder in the directory .ssh/ these files were generated ls ~/.ssh/ id_rsa id_rsa.pub * Send us our public SSH key, which is located in ''id_rsa.pub''. You can display it in the terminal by typing cat ~/.ssh/id_rsa.pub * Keep the private key ''~/.ssh/id_rsa'' secret! Do not send nor give it to anybody. ===Extended accounts=== The extended accounts allow users to use the extended MetaCloud testbed functionality. *''GridFTP account'' - you can apply for this account, if you have personal certificate used in MetaCentrum or its partners. This allows you for more effective manipulation with images and uploading those to the server. If you request a ''GridFTP account'', please send us your [http://en.wikipedia.org/wiki/Distinguished_Name#Directory_structure DN] from your x.509 personal/grid certificate. [[Category:English]] [[Category:Clouds]] j346587r5yhl71f1rg4rnxxqd0teb6g 0 504 1391 1390 2016-02-26T19:21:06Z Root 1 naimportována 1 revize wikitext text/x-wiki <span style="color:#ff0000">'''THIS PART OF WIKI IS OUTDATED AND UNDER CONSTRUCTION. PLEASE BE PATIENT.'''</span> ([[Přístup_do_MetaCloudu_přes_webové_rozhraní|Czech version]]) '''Prerequisite:''' [[MetaCloud_testbed_registration | OpenNebula account]] === OpenNebula Sunstone === OpenNebula Sunstone is a [https://carach1.ics.muni.cz/ web interface], that implements basic API for work with cloud. It provides functionality such as creating VM templates, instantiation of VMs and their management and monitoring. Supported web browsers are: Chrome/Chromium (full support, including in-browser VNC), latest versions of Opera, Safari and Firefox (partially supported). ====Tutorials==== * [[MetaCloud_testbed_-_quick_guide|Instantiation of a preconstructed VM image - quick guide]] * [[MetaCloud_management_via_web_interface|Detailed tutorial about OpenNebula Sunstone interface]] ====Warning==== Sunstone web interface is recommended for beginners mostly as a simple tool for basic work with cloud and its functionality is rather limited. Full functionality of MetaCloud can be achieved by using [[Přístup do Metacloudu přes příkazovou řádku|ONE tools]]. [[Kategorie:English]] [[Kategorie:Clouds]] 4rnugblv4t5bjdnqb0tjrpaw4j2rx9d 0 505 1393 1392 2016-02-26T19:21:06Z Root 1 naimportována 1 revize wikitext text/x-wiki <span style="color:#ff0000">'''THIS PART OF WIKI IS OUTDATED AND UNDER CONSTRUCTION. PLEASE BE PATIENT.'''</span> ([[MetaCloud_testbed_-_rychlý_průvodce|Czech version]]) __TOC__ This tutorial will guide you through starting the virtual machine (VM hereafter) '''prepared by us''' through [https://carach1.ics.muni.cz Sunstone web interface]. It contains some basic, but essential steps. '''Prerequisite''': You already have your [[MetaCloud testbed registration|basic account]], you sent us your public SSH key and you know your login and password from confirmation e-mail sent by us. ==Logging in through web interface== Log into [https://carach1.ics.muni.cz web interface]. Use your login and password from the confirmation e-mail. ==Choosing an image and starting a virtual machine== Go to section '''Templates''' from the left panel. Once there, choose yourself a template. Template is a collection of information about a potential virtual machine (network, amounts of RAM, source image file etc.). Along with your basic account we created [[Prepared images and templates|a set of images and a template file]] for you to try out (we can create templates from other images for you if you are interested). Images can be: * '''Persistent''' - the VM state is saved, you can only instantiate one VM from this kind of template (prepared) * '''Non persistent''' - the VM state is lost after shutdown, all the data is lost, the amount of instantiated VMs is not limited with this kind of template Check the chosen template and click '''Instantiate'''. After a short while the VM is running. Go to '''Virtual machines''' pane from the left panel and click [[Soubor:Refresh-icon.png|||]]. This refreshes the information and you can see your virtual machine(s). The IP address of your VM is located in tab '''VM Template''', accessible after you select your machine. Take a note of '''the IP of your VM'''. '''Please note''': The value in the column ''hostname'' is not the address of your VM. This value is the address of the physical machine, which is hosting your VM. You can also refer to extensive [[MetaCloud management via web interface|guide to the web interface]]. ==Logging in to the virtual machine== To log into the VM as ''root'' through ssh (we imported the ssh key into the VM for you) issue the following: ssh root@<the_IP_address_of_the_VM> [[Kategorie:English]] [[Kategorie:Clouds]] knz0qd5b5nx6pd1px6wemthqrpfem0u 0 506 1395 1394 2016-02-26T19:21:07Z Root 1 naimportována 1 revize wikitext text/x-wiki <span style="color:#ff0000">'''THIS PART OF WIKI IS OUTDATED AND UNDER CONSTRUCTION. PLEASE BE PATIENT.'''</span> ([[Správa_Metacloudu_přes_webové_rozhraní|Czech version]]) ==Starting the virtual machine== All the available templates and images can be found under the ''Virtual Resources'' -> ''Templates'' or ''Images'' sections in the left menu. When in ''Templates'' section you can choose one of the templates and initiate a virtual machine by clicking on the ''Initiate'' button in the top right menu. List of your running virtual machines can be found undre the ''Virtual Machines'' section.You should pay attention to the ''Status'' column where you can see the current virtual machine's state. There is also a ''VM's IP(s)'' column which contains an VM's address for ssh connection. {| | [[Soubor:Images_web.png|left|thumb|List of the available images]] | [[Soubor:Templates_web.png|left|thumb|List of the available templates]] | [[Soubor:Machines_web.png|left|thumb|List of the running VMs]] |} ==Metacloud management== ===VM templates management=== List of the available templates can be found ''Virtual Resources'' -> ''Templates'' section in the left menu. When in ''Templates'' section you can choose one of the templates and initiate a virtual machine by clicking on the ''Initiate'' button in the top right menu. Clic on the template name to see its information and click on the ''Template'' button to see aditional information. [[Soubor:Templates_adv_01_web.png|none|thumb|Template section]] ====New template creation==== You can also create your own template file. Seek the 'Virtual Resources -> Templates' tab (1.) and click on '+New' button (2.). 'Create VM template' window will pop up. Switch to 'Wizard XEN' tab (3.). Now just fill in following parameters and the template file will be ready. [[Soubor:cloud_template01.png|right|thumb|(1) Path to creating new template file]] [[Soubor:template02.png|right|thumb|(2) Setting example]] [[Soubor:template03.png|right|thumb|(3) Setting example cont.]] *'''Capacity options''' **Name - name of the template, that will be shown in templates list **Memory - memory size of virtual machine in MB (eg. 512, current memory maximum for VM is exactly 94647MB) **CPU - number of VM's VCPUs you plan to use (it is possible set half a CPU ie. 0.5, but if you plan CPU demandig computations, please, set it to same number as VCPU) **VCPU - number of virtual CPUs (if not set, default settings will be used) *'''Boot/OS options''' **Architecture - CPU architecture **Boot method - boot method, choose bootloader **Bootloader - set to pygrub *'''Add disks/images''' **Image - choose wanted image from options, eg. debian6 **Target - device name undre the system **Driver - disk driver for XEN **after setting up all the parameters, click on the 'Add' button and disk will be added to 'Current disks' list *'''Setup Networks''' **Network - from given options choose the network named 'public' **IP - leave out blank (there is no need of special IP address) **[[#Firewall|Firewall setup]] is described below. **Click on the 'Add' button, and in the 'Current NICs' list should appear a network with ID 1. *'''Add context variables''' **Set up this parameter only if you want the VM to be contextualized. **Sequentially enter pairs 'Name' and 'Value' and confirm each by clicking the 'Add' button. The pair will show up in 'Current variables' list. **example: Name - ssh_key Value - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUhViIU1WmVFnQ THwgAYg+u5lglCZb5nhJvdlFoNoKCdxxTcCEnLDhWH+lgK1vK5N1RL0wEIdgB 7YHnXcc5WHSBRrQ2BWhUI+vMDa0RDs3hXWp9NwBtHYSUj2P73bp+Zq4QPTYRW ISjJvWXKRjM9fqx24ffrTWtWSiDjyWexe9ZygNkc6UIuOolNgGrg90TzHILtD 2UINqAUQXEs2sxx1Yi88MFNRSDvcjyzRQv/2BTBsFXIwScuI6cOn9Gp0pxpVi wiUSIaZBcJBzF2SpUf4SAKNn9q53/Q2R6SphWUzKuEDIxmDrZkwKiqqMjq1aa JPb2FzVTnTbF4xgGNQH+Hjf login@host Name - files Value - http://is.muni.cz/www/256254/init.sh init.sh is sample contextualization script, which however must be supported by VM. Contextualization is optional. Name - hostname Value - DebianVM Remaining options are optional as well so you can leave default values in there. By clicking the 'Create' button the template will be created and it will be shown in the list. ====Modifying a template==== Instead of creating a new template you can also copy one of teh prepared templates and modify it according to you. In ''Template'' section choose the template you want to cop and click on the ''Clone'' button in the top right menu. In clone dialog fill the template name and click ''Clone''. You can modify your new template by clicking on teh ''Update properties'' button in the top right menu. To change the system disk (how to save a system disk is described in section [[#Disk management|Disk management]]) in a template you should update these settings: DISK=[ DEV_PREFIX="vd", IMAGE="name of saved image", IMAGE_UNAME="your user name" ] [[Soubor:Templates_adv_02_web.png|none|thumb|Template clone and modify]] ===VM images management=== List of available images can be found undre the ''Virtual Resources'' -> ''Images'' section. You can [[Uploading_own_images_to_MetaCloud|upload your own images]] to MetaCloud or copy one of the available images. Choose the image and click on the ''Clone'' button in the top right menu. Choose the image name and click ''Clone''. You can change the persistence of your image by clicking on the ''Persistent'' column of your image. Data stored on persistent image won't be deleted after the shutdown and you can initiate only one VM with persistent image. On the other hand non-persistent image won't store data aftre the shutdown but you can initiate more then one VMs with this image. For better understanding see [[MetaCloud testbed - workflow example|Cloud workflow example]]. {| | [[Soubor:Images_web.png|left|thumb|List of the available images.]] | [[Soubor:Images_copy_web.png|left|thumb|Image cloning.]] |} ===VM management=== [[Soubor:cloud_control01.png|right|thumb|450px|(1) Virtual Machines]] To manage VMs via Sunstone interface you have to switch to 'Virtual Machines' tab (1.) first. There you can see the list of running VMs, that are visible for you. For more information about particular VM, you just have to click on it in the list (2.) and in the bottom part of the screen (3.) the information will show divided into several tabs: *'''VM information''' *'''Disks & Hotplugging''' - disk management, discribed below *'''VM template''' - detailed info about the template file, including the IP address generated by OpenNebula *'''VM log''' *'''History information''' *'''Monitoring information''' More actions can be done after selecting VM (4.). After that the select box (6.) will be available with these options: *'''Hold''' - Switch the status of VM from PENDING to HOLD. *'''Release''' - Release the VM from HOLD status. *'''Suspend''' - Switch the selected VMs to SUSPENDED status. *'''Resume''' - Resume VMs from the STOPPED or SUSPENDED status. *'''Stop''' - Stops selected VMs. *'''Restart''' - Restarts selected VMs in the UNKNOWN or BOOT status. *'''Resubmit''' - Resubmits selected VMs in the PENDING status. *'''Save as''' - OpenNebula usually does not save current state of VM after shutdown, therefore if you want to save the changes made in VM you have to do so with this action before using 'Shutdown' (6.). New image will be saved and new template has to be created, because the ID of the image will be different.<br/>Note: after using 'Save as' it is important to turn the VM off with 'Shutdown'. Turning off the VM by using 'Cancel' or 'Delete' will cancel the 'Save as' action. *'''Cancel''' - Cancels selected VMs The complete list of possible VMs' states (described in detail in [http://www.opennebula.org/documentation:rel3.2:vm_guide_2#virtual_machine_life-cycle documentation]) *Pending (pend): Waiting for resources and deployment. *Hold (hold): The owner has held the VM and it will not be scheduled until it is released. It can be, however, deployed manually. *Prolog (prol): The system is transferring the VM files (disk images and the recovery file) to the host in which the virtual machine will be running. *Running (runn): The VM is running. In this state, the virtualization driver will periodically monitor it. *Migrate (migr): The VM is migrating from one resource to another. *Epilog (epil): In this phase the system cleans up the Host used to virtualize the VM, and additionally disk images to be saved are copied back to the front-end. *Stopped (stop): The VM is stopped. VM state has been saved and it has been transferred back along with the disk images to the front-end. *Suspended (susp): Same as stopped, but the files are left in the host to later restart the VM there. *Failed (fail): The VM failed. *Unknown (unknown): The VM couldn't be reached, it is in an unknown state. *Done (done): The VM is done. VMs in this state won't be shown with “onevm list” but are kept in the database for accounting purposes. ==Firewall== '''Warning:''' Users are responsible for the security of their virtual machines. Please pay attention to correct firewall setup. All VMs have firewall turned on by default. The only opened ports are TCP 22 and UDP 67, 68. Other ports can be opened in two ways. During the template creation: In addition to basic method of template creation, described above, before you click on 'Add' button, select Tcp or Udp firewall mode (Port whitelist) in section 'Setup Networks'. Then you write the list of ports into the shown box: &bull; Different ports divided by comma,<span style="color:red;"> '''without spaces'''</span>, eg. 68,67,69 &bull; Range of ports using colon, eg. 6000:6500 &bull; Or combination of both, eg. 68,69,6000:6500 By editing an existing template: In 'Templates' tab select template you want to edit and click on 'Update a template' button. Edit the ports in opened window ('''in quotes and without spaces'''): &bull; Different ports divided by comma,<span style="color:red;"> '''without spaces'''</span>, eg. "68,67,69" &bull; Range of ports using colon, eg. "6000:6500" &bull; Or combination of both, eg. "68,69,6000:6500" eg.: NIC=[ NETWORK_ID=1, WHITE_PORTS_TCP="22,443,8443,8080,8180", WHITE_PORTS_UDP="67,68" ] ==Disk management== [[Soubor:hotplugging.png|right|thumb|Hotplugging]] Disks in virtual machines can be attached or detached while the VM is still running (hotplugging). You cannot attach the system disks, only the data disks. To change the system disk in a VM you should [[#Modifying a template|modify a template]] of an existing machine. In 'Virtual Machines' tab (1.) select the virtual machine, in which you want to add or remove the disk (2.) and switch to the 'Disks & Hotplugging' tab (3.). To remove the disk just click the 'Detach X' button (4.) next to the selected disk. There are two options how to attach a disk to VM: *If you have a prepared image select 'Type: Existing image' and select your image in 'Select image:' menu. 'Device prefix:' should be '''vd''' and then just click the 'Attach' button (5.). *Otherwise select 'Type: Volatile disk', enter size of the disk, its format e.g. '''ext3''' and 'Type:' fs (file system) or swap. 'Device prefix:' should be '''vd''' and then just click the 'Attach' button (5.). '''The disk in VM will show up in /dev as DEVICE_PREFIX + {c, d, e, ...}. Suffix letter is generated alphabetically used as first free letter!''' '''It is recommended to not change /etc/fstab! Rather modify template for stable disk connection.''' '''Do not use /dev/vdb as connection point as in template as in /etc/fstab! This drive is reserved for contextualization and VM will be unbootable otherwise! [[Category:English]] [[Category:Clouds]] guzt1qltfbu99ri6nyt2z4xwtjht7m4 0 507 1397 1396 2016-02-26T19:21:07Z Root 1 naimportována 1 revize wikitext text/x-wiki <span style="color:#ff0000">'''THIS PART OF WIKI IS OUTDATED AND UNDER CONSTRUCTION. PLEASE BE PATIENT.'''</span> ([[Nahrání_vlastních_obrazů_do_MetaCloudu |Czech version]]) == How to upload your own images? == '''Prerequisite:''' you have OpenNebula client tools configured correctly ([[MetaCloud access through command line]]). This tutorial will show you how to upload your own image into MetaCloud. Before you can work with your own OS images via OpenNebula's web interface Sunstone or command line utilities ONE tools, you have to upload the images to the storage. === GridFTP (preferred) === '''Prerequisite:''' You have client grid certificate and key in pem format, you requested GridFTP account, you have successfully installed CA from IGTF. This method requests your grid certificate present in your PC and a knowledge of GridFTP. If you are using certificate in Metacentrum this would probably be the easiest way to upload your image. This method uploads image directly to our server, where you can either via Sunstone interface or ONE tools add it to MetaCloud. *Create the folder ~/.globus, put your own client certificates. ~/.globus/usercert.pem # x509 certificate, chmod 644 ~/.globus/userkey.pem # rsa private key, no password, chmod 600 *Install the certificates of trusted authorities in /etc/grid-security/certificates, MetaCloud uses certificates signed by TERENA eScience SSL CA. You can use this tutorial [[FedCloudDocumentation#Installation]]. Initialize client proxy certificate grid-proxy-init (there is -debug parameter for basic problem debugging) Copy the image to storage server (which is different from MetaCloud storage server) globus-url-copy file:///<ABSOLUTE_PATH_TO_THE_IMAGE_ON_YOUR_DISK>/<FILE> gsiftp://carach4.ics.muni.cz:50000/home/<YOUR_LOGIN>/ '''Warning: Do not forget the slash in the end of the command!''' List storage space globus-url-copy -list gsiftp://carach4.ics.muni.cz:50000/home/<YOUR_LOGIN>/ If this error occurs globus_gsi_gssapi: SSLv3 handshake problems: Couldn't do ssl handshake OpenSSL Error: s3_clnt.c:2985: in library: SSL routines, function SSL3_SEND_CLIENT_VERIFY: EVP lib OpenSSL Error: rsa_sign.c:127: in library: rsa routines, function RSA_sign: digest too big for rsa key please re-generate the proxy certificate with RSA key at least 1024 bits long (implicit 512 bits is not enough) grid-proxy-init -bits 1024 #or 2048 a) If you want to use [https://carach1.ics.muni.cz web interface] to upload the image from storage to MetaCloud, just follow these steps. [[Soubor:Image gridftp2.png|right|thumb|200px|(Img. 2) GridFTP - Preferred method]] The image uploaded with GridFTP can be added to MetaCloud in menu Images (1) -> New (2) (Img. 1). Next check option "Provide a path" (3) and enter the path to your private GridFTP folder and name of the image you uploaded there (4) (Img. 2). [[Soubor:Image gridftp1.png|center|thumb|300px|(Img. 1) GridFTP - Preferred method]] b) If you want to use ONE tools to upload the image from storage to MetaCloud, follow the [[MetaCloud management via command line]]. cat debian.one NAME = "Debian6" PATH = /home/<YOUR_LOGIN>/<FILE> #Path in storage server TYPE = OS DESCRIPTION = "Your description of OS image" oneimage create debian.one ID: 0 === Publicly available image (alternative) === [[Soubor:Image gridftp3.png|right|thumb|200px|(Img. 4) Publicly available image]] You also can upload your image to MetaCloud if it is available in public hosting. It must however: *Be in RAW format *Not be compressed *And the link must begin with http://, not https:// To add the image click on Images (1) -> New (2) (Img. 3). Next check the option "Provide a path" (3) and enter the link to your image (4) (Img. 4). [[Soubor:Image gridftp1.png|center|thumb|300px|(Img. 3) Publicly available image]] === Upload in Sunstone === [[Soubor:ONuploadImage.png|right|thumb|200px|(Obr. 6) Upload of local image in Sunstone]] To add the image click on Images(1) -> New (2) (Img. 5). Then fill in the name, check the "Upload (3)" option and set the path to your local image file on your hard drive (4) (Img. 6).<br> '''Warning:''' The size of the image uploaded in Sunstone is limited to 10GiB. In addition, there might be some upload size restrictions in your browser. [[Soubor:Image gridftp1.png|center|thumb|300px|(Obr. 5) Upload in Sunstone]] === OCCI interface (alternative) === The most effective way to upload your image is by using OCCI interface. It is implemented by utilities: *occi-compute *occi-network *occi-storage We are going to use the last one. But first you have to create a text file with XML content &ndash; template, which contains certain information about uploaded image. cat debian.occi <STORAGE> <NAME>Name of your image</NAME> <DESCRIPTION>Description of your image</DESCRIPTION> <TYPE>OS</TYPE> <URL>file:///<ABSOLUTE_PATH_TO_THE_IMAGE>/<FILE></URL> </STORAGE> Uploading the image after creating the template is just a matter of one command. occi-storage create debian.occi Output of this command after successful upload should look like this: <STORAGE href='https://carach1.ics.muni.cz:9443/storage/0'> <ID>0</ID> <NAME>Name of your image</NAME> <TYPE>OS</TYPE> <DESCRIPTION>Description of your image</DESCRIPTION> <PUBLIC>NO</PUBLIC> <PERSISTENT>NO</PERSISTENT> <SIZE>41943040</SIZE> </STORAGE> You can also view the list of available images. occi-storage list <STORAGE_COLLECTION> <STORAGE href='carach1.ics.muni.cz/storage/'''0'''' name='suse-test'/> <STORAGE href='carach1.ics.muni.cz/storage/'''9'''' name='Ubuntu_raw_test'/> </STORAGE_COLLECTION> Or you can delete the image according to its ID. occi-storage delete 9 === EC2 API (alternative)=== If needed, it is possible to use alternative way to upload your image. This way however is not recommended, the image can not be named, because the name is generated automatically. On the other hand it is the fastest way and you do not have to create any template. econe-upload /<ABSOLUTE_PATH_TO_THE_IMAGE>/<FILE> Success: ImageId ami-00000001 == How to check the successful upload? == Again we can use either [https://carach1.ics.muni.cz web interface], where everything is clearly shown in "Images" section. Or ONE tools by entering a simple command: $ oneimage list ID USER GROUP NAME SIZE TYPE REGTIME PER STAT RVMS 0 <YOUR_LOGIN> users Debian 4G OS 01/13 07:36:49 No rdy 0 ==MS Windows== Using the images of Microsoft Windows operating system is limited because of the possible problems with license. Nevertheless if you are interested in using such images in our cloud, please contact us at [mailto:cloud@metacentrum.cz cloud@metacentrum.cz]. [[Kategorie:English]] [[Kategorie:Clouds]] 7d8giahgvk4uxodzrmw7ok6erdmwn16 0 508 1399 1398 2016-02-26T19:21:07Z Root 1 naimportována 1 revize wikitext text/x-wiki <span style="color:#ff0000">'''THIS PART OF WIKI IS OUTDATED AND UNDER CONSTRUCTION. PLEASE BE PATIENT.'''</span> ([[Správa_MetaCloudu_pomocí_příkazové_řádky|Czech version]]) == VM instantiation == List of all templates and images available to you. $ oneimage list ID USER GROUP NAME DATASTORE SIZE TYPE PER STAT RVMS 8 oneadmin users debian6 default 8G OS No rdy 0 41 oneadmin users scilinux5 default 10G OS No rdy 0 42 oneadmin users scilinux6 default 10G OS No rdy 0 $ onetemplate list ID USER GROUP NAME REGTIME 5 oneadmin users debianvm 02/15 18:27:09 55 oneadmin users scilinux5vm 04/24 13:04:17 56 oneadmin users scilinux6vm 04/24 13:05:04 Now you can instantiate a VM. $ onetemplate instantiate 5 VM ID: 1977 This VM will show up in the list of running VMs. Important is the STAT column, where you can find the information about the state of the VM. $ onevm list ID USER GROUP NAME STAT UCPU UMEM HOST TIME 1977 <YOUR_LOGIN> users one-1977 pend 0 0K 0d 00h00 $ onevm list ID USER GROUP NAME STAT UCPU UMEM HOST TIME 1977 <YOUR_LOGIN> users one-1977 runn 0 0K dukan3.ics 0d 00h00 List additional information about running VM ("runn" in STAT column). $ onevm show 1977 VIRTUAL MACHINE 1977 INFORMATION ----------------------------- ... NIC=[ BRIDGE=xenbr0, CLUSTER_ID="100", IP=<IP_ADRESA_VM>, # address for ssh connection to VM MAC=02:00:c0:a8:fe:03, NETWORK=public, NETWORK_ID="1", VLAN="NO", WHITE_PORTS_TCP="22", WHITE_PORTS_UDP="67,68" ] ... == Metacloud management == Here is the list of all programs which implement ONE API. Each command can be reviewed more closely by using -h switch. Next there are some important commands for work in MetaCloud via command line (ONE utils). Theyr meaning is intuitive. * oneacl: access control management * oneuser: add, remove and monitor users * onegroup: group access control management * onehost: add, remove and monitor computing hosts * oneimage: add, remove and manage images * onetemplate: add, remove and manage templates of VMs * onevm: create, control and monitor VM * onevnet: add, remove and monitor virtual networks * onezone: zone management * onevdc: data center management ====VM templates management:==== List of available templates $ onetemplate list ID USER GROUP NAME REGTIME 5 oneadmin users debianvm 02/15 18:27:09 55 oneadmin users scilinux5vm 04/24 13:04:17 56 oneadmin users scilinux6vm 04/24 13:05:04 Initiation of VM $ onetemplate instantiate debianvm --name DebianVM VM ID: 1977 You can create your own template and uploud it to MetaCloud database. $ cat DebianVM.one #--------------------------------------- # VM definition example #--------------------------------------- NAME = DebianVM CPU = 1 MEMORY = 1024 # --- boot options --- OS=[ BOOTLOADER=pygrub ] RAW=[ TYPE=xen ] # --- 3 disks --- # TARGET xvdb is reserved for contextualization of your VM DISK = [ IMAGE_ID = <IMAGE_ID>, DRIVER = "tap2:tapdisk:aio:", TARGET = "xvda" ] DISK = [ TYPE = swap, SIZE = 1024, READONLY = "no", TARGET = "xvdc" ] DISK = [ TYPE = fs, SIZE = 4096, FORMAT = ext3, SAVE = no, TARGET = xvdd ] # --- 1 NIC --- # Own network creation is forbidden, # use the default network created by us. NIC = [ NETWORK_ID = 1, WHITE_PORTS_TCP="22", WHITE_PORTS_UDP="67,68" ] # --- Placement options --- REQUIREMENTS = "CPUSPEED > 1000" RANK = FREECPU # --- Contextualization --- # Only in case you need contextualization CONTEXT = [ FILES = "http://is.muni.cz/www/256254/init.sh", SSH_KEY = "$USER[SSH_KEY]", HOSTNAME = "MyVM", # bude prepsano nslookup $PUBLIC_IP PUBLIC_IP = "$NIC[IP]" ] $ onetemplate create DebianVM.one ID: 583 Or you can copy some of available tempolates and modify it. $ onetemplate clone debianvm MyTemplate ID: 584 $ onetemplate update 584 ====VM images management:==== List of available images $ oneimage list ID USER GROUP NAME DATASTORE SIZE TYPE PER STAT RVMS 8 oneadmin users debian6 default 8G OS No rdy 0 41 oneadmin users scilinux5 default 10G OS No rdy 0 42 oneadmin users scilinux6 default 10G OS No rdy 0 You can [[Uploading_own_images_to_MetaCloud|upload your own images]] to MetaCloud or copy one of the available images. $ oneimage clone debian6 MyImage ID: 371 You can change the persistence of your image. $ oneimage persistent 371 $ oneimage nonpersistent 371 Data stored on persistent image won't be deleted after the shutdown and you can initiate only one VM with persistent image. On the other hand non-persistent image won't store data aftre the shutdown but you can initiate more then one VMs with this image. For better understanding see [[MetaCloud testbed - workflow example|Cloud workflow example]]. ====VM management:==== $ onevm list ID USER GROUP NAME STAT CPU MEM HOSTNAME TIME 0 <YOUR_LOGIN> users DebianVM pend 0 0K 00 00:00:03 $ onevm list ID USER GROUP NAME STAT CPU MEM HOSTNAME TIME 0 <YOUR_LOGIN> users DebianVM runn 0 0K dukan1.ics.muni.cz 00 00:02:40 $ onevm suspend 0 $ onevm list ID USER GROUP NAME STAT CPU MEM HOSTNAME TIME 0 <YOUR_LOGIN> users DebianVM susp 0 512K dukan1.ics.muni.cz 00 00:11:13 $ onevm resume 0 $ onevm list ID USER GROUP NAME STAT CPU MEM HOSTNAME TIME 0 <YOUR_LOGIN> users DebianVM runn 0 512K dukan1.ics.muni.cz 00 00:15:13 $ onevm stop 0 $ onevm list ID USER GROUP NAME STAT CPU MEM HOSTNAME TIME 0 <YOUR_LOGIN> users DebianVM stop 0 512K dukan1.ics.muni.cz 00 00:32:13 $ onevm resume 0 $ onevm list ID USER GROUP NAME STAT CPU MEM HOSTNAME TIME 0 <YOUR_LOGIN> users DebianVM pend 0 512K dukan1.ics.muni.cz 00 00:34:13 The complete list of possible VMs' states (described in detail in [http://www.opennebula.org/documentation:rel3.2:vm_guide_2#virtual_machine_life-cycle documentation]) *Pending (pend): Waiting for resources and deployment. *Hold (hold): The owner has held the VM and it will not be scheduled until it is released. It can be, however, deployed manually. *Prolog (prol): The system is transferring the VM files (disk images and the recovery file) to the host in which the virtual machine will be running. *Running (runn): The VM is running. In this state, the virtualization driver will periodically monitor it. *Migrate (migr): The VM is migrating from one resource to another. *Epilog (epil): In this phase the system cleans up the Host used to virtualize the VM, and additionally disk images to be saved are copied back to the front-end. *Stopped (stop): The VM is stopped. VM state has been saved and it has been transferred back along with the disk images to the front-end. *Suspended (susp): Same as stopped, but the files are left in the host to later restart the VM there. *Failed (fail): The VM failed. *Unknown (unknown): The VM couldn't be reached, it is in an unknown state. *Done (done): The VM is done. VMs in this state won't be shown with “onevm list” but are kept in the database for accounting purposes. ==Firewall== '''Warning:''' Users are responsible for the security of their virtual machines. Please pay attention to correct firewall setup. All VMs have firewall turned on by default. The only opened ports are TCP 22 and UDP 67, 68. Other opened ports can be defined in template in NIC section: &bull; Different ports divided by comma,<span style="color:red;"> '''without spaces'''</span>, eg. "68,67,69" &bull; Range of ports using colon, eg. "6000:6500" &bull; Or combination of both, eg. "68,69,6000:6500" eg.: NIC=[ NETWORK_ID=1, WHITE_PORTS_TCP="22,443,8443,8080,8180", WHITE_PORTS_UDP="67,68" ] ==Disk management== New disks can be hot-plugged to running VMs with the onevm attachdisk and detachdisk commands. For example, to attach to a running VM the Image named “storage”: $ onevm attachdisk one-5 --image storage To detach a disk from a running VM, find the disk ID of the Image you want to detach using the onevm show command, and then simply execute onevm detach vm_id disk_id: $ onevm show one-5 ... DISK=[ DISK_ID="1", ... ] ... $ onevm detachdisk one-5 1 [[Category:English]] [[Category:Clouds]] jz36c6ufbgdovisrgoflsunz9s62iqt 0 509 1401 1400 2016-02-26T19:21:07Z Root 1 naimportována 1 revize wikitext text/x-wiki <span style="color:#ff0000">'''THIS PART OF WIKI IS OUTDATED AND UNDER CONSTRUCTION. PLEASE BE PATIENT.'''</span> ([[Podpora obrazů a konverze|Czech version]]) In order to use your own images in OpenNebula, they have to be in supported format. == What are the supported OS images and how to convert them? == Your OS images will run under virtualization system Xen v4.0.1, they must be therefore compatible with this system. Easiest way to check the compatibility is to use utility "file" in unix-type OS. Command "file image_file_name" should output: *if it is RAW image of whole disk, ie. more partitions with MBR and bootloader ('''preferred format''') linux-diskimage.img: x86 boot sector; GRand Unified Bootloader, stage1 version 0x3, boot drive 0x80, 1st sector stage2 0x849f2, GRUB version 0.94; partition 1: ID=0x83, active, starthead 32, startsector 2048, 1024000 sectors; partition 2: ID=0x8e, starthead 221, startsector 1026048, 5265408 sectors, code offset 0x48 or *if it is RAW image of filesystem ('''for advanced users'''). It is easier to create, but less flexible and interoperable. linux-rootfsimage.img: Linux rev 1.0 ext3 filesystem data, UUID=81a2e5b2-8d1f-4205-9de9-c47ff1186a23, volume name "sci-linux" (large files) If the output is different from those above or the image format is different (eg. QCOW(2) format), you have to convert it into RAW format to use it in MetaCloud (eg. with "qemu-img convert"). Tutorial for advanced users can be found in [[FedCloudDocumentation#How_to_convert_your_image_to_OpenNebula-compatible_format.3F]] (EN). '''Note:''' The kernel of your image must also support Xen. It must have xen in it's name (*-2.6.32-5-xen-amd64) or must be 3.0 or higher. '''Note:''' If you experience problems with damaged ext4 journal after restart, add option barrier=0 to / in /etc/fstab [[Category:English]] [[Category:Clouds]] dj77skavzsgkar2zy15p5m73i95fnuh 0 510 1403 1402 2016-02-26T19:21:08Z Root 1 naimportována 1 revize wikitext text/x-wiki <span style="color:#ff0000">'''THIS PART OF WIKI IS OUTDATED AND UNDER CONSTRUCTION. PLEASE BE PATIENT.'''</span> ([[Změna hesla do MetaCloudu|Czech version]]) '''Watch out! If used incorrectly these commands may block your account.''' '''Web interface''' Password can be also changed through [https://carach1.ics.muni.cz/ web interface]. [[Soubor:Change_passwd_net.png|center|thumb|300px|(Pic. 1) Change password via web interface]] '''From command line''' Supposing you have your environment set up correctly, display your unique ID using: oneuser show USER 4 INFORMATION '''ID : 28''' NAME : your_name GROUP : 1 PASSWORD : XXXXXXXXXXXXXXXXXXXXXXXXXXXX ENABLED : Yes Upon getting your ID, you can change your password with the following: oneuser passwd '''<your_ID>''' <your_new_password> Do not forget to update the password in the authentication file ''~/.one/one_auth''. [[Kategorie:English]] [[Kategorie:Clouds]] 658hlfdsp9q0h38c1won4zeu2pr6s86 0 511 1405 1404 2016-02-26T19:21:08Z Root 1 naimportována 1 revize wikitext text/x-wiki ([[Úložiště_Cumulus|Czech version]]) '''Cumulus is no longer supported.''' [[Kategorie:Clouds]] [[Kategorie:Help]] [[Kategorie:English]] 4kczpf5s3jee8efoitkwz58ggryqcc1 0 512 1407 1406 2016-02-26T19:21:08Z Root 1 naimportována 1 revize wikitext text/x-wiki <span style="color:#ff0000">'''THIS PART OF WIKI IS OUTDATED AND UNDER CONSTRUCTION. PLEASE BE PATIENT.'''</span> ([[MetaCloud testbed - příklad práce s cloudem|Czech version]]) For better understanding of how to work with cloud, we prepared a simple use case for you, where we will illustrate the basic functionality of MetaCloud. ===Web interface usage example=== The easiest way is to use one of the templates and images we prepared for you. You can find their lists in the ''Virtual Resources'' section in the left menu within the section ''Templates'' and ''Images''. Choose the template you want to start and click on the ''Initiate'' button in the top right menu.Now, you can connect via ssh to your virtual machine which address you can find in column ''VM's IP(s)''. All our virtual machines' images are non-persistent. That means all data stored on the virtual machine are lost after its shutdown. If you need a virtual machine which keep data aftre its shutdown you have to upload your own image to MetaCloud [[[[Uploading_own_images_to_MetaCloud|upload image]] or copy one of our images. You can do that in images section by clicking on the ''Clone'' button in the top right menu. Now you can set your image to persistent in ''Persistent'' column. Don't forget that your new image has different ID (''ID'' column) so you have to support it with right template. You can create a new one or you can copy and modify one of our prepared templates. Again you can choose the template from the list and click on ''Clone'' button in the top right menu. Click on the ''Update properties'' button and you will be shown a dialogue with templates properties. Find the row ''IMAGE_ID'' and set to the ID of your new image. Now when you start the virtual machine with your peristent image all chages and data will be stored aftre the machine shutdown. You can start only one virtual machine with persistent image. If you need more virtual machines with this modified image, change the image back to non-persistent and than you can run more of them. {| | [[Soubor:Images_web.png|left|thumb|List of available images]] | [[Soubor:Templates_web.png|left|thumb|List of available templates]] | [[Soubor:Images_copy_web.png|left|thumb|Copy image]] | [[Soubor:Templates_adv_02_web.png|left|thumb|Copy and modify template]] |} ===Command line usage example=== We suppose you have correctly configured and functioning environment with OpenNebula [[MetaCloud_access_through_command_line|client tools]]. The easiest way is to use one of the templates and images we prepared for you. $ oneimage list ID USER GROUP NAME DATASTORE SIZE TYPE PER STAT RVMS 8 oneadmin users debian6 default 8G OS No rdy 0 41 oneadmin users scilinux5 default 10G OS No rdy 0 42 oneadmin users scilinux6 default 10G OS No rdy 0 $ onetemplate list ID USER GROUP NAME REGTIME 5 oneadmin users debianvm 02/15 18:27:09 55 oneadmin users scilinux5vm 04/24 13:04:17 56 oneadmin users scilinux6vm 04/24 13:05:04 Now you can start a virtual machine. $ onetemplate instantiate 5 VM ID: 1977 When the machine is running you can establish a connection via ssh. $ onevm show 1977 VIRTUAL MACHINE 1977 INFORMATION ----------------------------- ... NIC=[ BRIDGE=xenbr0, CLUSTER_ID="100", IP=<IP_ADRESA_VM>, # address for SSH connction with VM MAC=02:00:c0:a8:fe:03, NETWORK=public, NETWORK_ID="1", VLAN="NO", WHITE_PORTS_TCP="22", WHITE_PORTS_UDP="67,68" ] ... All our virtual machines' images are non-persistent. That means all data stored on the virtual machine are lost after its shutdown. If you need a virtual machine keep data aftre its shutdown you have to upload your own image to MetaCloud [[[[Uploading_own_images_to_MetaCloud|upload image]] or copy one of our images. $ oneimage clone debian6 MyImage ID: 371 Now you can set image persistence. $ oneimage persistent 371 Don't forget that your new image has different ID (''ID'' column) so you have to support it with right template. You can create a new one or you can copy and modify one of our prepared templates. $ onetemplate clone debianvm MyTemplate ID: 584 $ onetemplate update 584 Find the row ''IMAGE_ID'' and set to the ID of your new image. Now when you start the virtual machine with your peristent image all chages and data will be stored aftre the machine shutdown. You can start only one virtual machine with persistent image. If you need more virtual machines with this modified image, change the image back to non-persistent and than you can run more of them. [[Category:English]] [[Category:Clouds]] p4tbwo4jg4z6pwh5y5vl1hwxws7zbtk 0 513 1409 1408 2016-02-26T19:21:08Z Root 1 naimportována 1 revize wikitext text/x-wiki <span style="color:#ff0000">'''THIS PART OF WIKI IS OUTDATED AND UNDER CONSTRUCTION. PLEASE BE PATIENT.'''</span> ([[Známé problémy v MetaClodu a jejich řešení|Czech version]]) This page is dedicated to solving problems, that can occur in the MetaCloud. If you encounter a problem which is not described here, please [mailto:cloud@metacentrum.cz contact us] and we will be at your service. [[Kategorie:English]] [[Kategorie:Clouds]] s495glac53hd9rv4pjyqyk4fsn2iyyx 0 514 1411 1410 2016-02-26T19:21:08Z Root 1 naimportována 1 revize wikitext text/x-wiki ([[Cloudová bezpečnostní politika|Czech version]]) This document describes operational rules imposed by CESNET on its cloud facilitites. It documents technical measures developed to achieve a reasonable level of operational security of the cloud service. The main goal of this document is to define measures and rules to limit security risks posed by cloud applications and users, and to provide feasible mechanisms to ease resolution of security incident that might happen in the cloud facilities. The document was built with the existing [https://www.egi.eu/about/policy/policies_procedures.html EGI security policies] in mind. Whenever applicable, we refer to existing policies to define the scope needed. For example, whenever 'image endorsement' is mentioned, we refer to the [https://documents.egi.eu/document/771 Security Policy for the Endorsement and Operation of Virtual Machine Images]. Other policies policies, however, deal with "classic" sites, and the cloud technology presents some additional facts that are not taken into account. For instance, the cloud provider (site) cannot guarantee proper patch management of the systems (since the machines might not operated directly by the site people). We are aware of the new class of problems, though, and we realize the problems may lead to security incidents that may differ from what we face with classic grid sites. We expect some new policies will be formed. For example, from the user's point point we are missing a policy similar to the [https://documents.egi.eu/document/84 Policy on Grid Pilot Jobs], which would define e.g. the responsibilities of VM owners. As an interim step however, we decided to specify measures defined to facilitate efficient security operations and streamline the incident response and users' traceability to a level which is feasible in our cloud environment. == Security measures == === Pre-runtime measures === ==== Endorsed images ==== * endorsements for virtual machine images implemented ** directly, as cryptographically signed hashes ** indirectly, based on verbal agreements * only virtual machine instances based on endorsed images are allowed to have public IP addresses * modified and subsequently saved images are no longer considered to be endorsed by the original endorser ==== Trusted users ==== * trusted users defined as users with ** high-level identity verification ** explicit endorsement from other trusted users or site managers * only trusted users have access to pools of public IP addresses ==== Restricted remote access to running virtual machines ==== * only the following combinations of access methods and authentication methods are allowed ** SSH with public key authentication ** SSH with GSS API authentication ** Encrypted RDP/VNC * password-based remote authentication methods are not allowed (e.g. SSH with a plain password) ==== Automated pre-runtime compliance testing ==== * all virtual machine images and virtual machine instances based on said images must be tested for explicit compliance with the defined security profile ([[Cloud_Security_Policy#Security_Profile]]) * only compliant images and virtual machine instances based on said images can ** be published (made available to other users) ** be assigned public IP addresses ** be launched outside isolated private networks === Runtime measures === ==== Networking insolation for L2 ==== * running virtual machine instance will be isolated in a VLAN if ** image the instance is based on is not endorsed by a trusted user ** it does not belong to a trusted user ** it is running OS Windows ** its owner chooses to isolate it ==== Networking isolation for L3 ==== * running virtual machine instance will be isolated using firewall if ** it has a public IP address ** its owner chooses to isolate it in a private network ==== IP logging ==== * every IP address given to a virtual machine instance will be tied to its owner for the duration of its lifetime (i.e. until shutdown) * owner of the virtual machine instance is responsible for any illegal activity during its lifetime ==== Anti-spoofing rules for networking ==== * network addresses assigned to a virtual machine instance by the cloud platform are mandatory and cannot be changed by the owner at runtime * anti-spoofing rules are enforced by the hypervisor or local network infrastructure * an attempt to change the assigned network addresses will immediately cut off the virtual machine instance from any subsequent network communication ==== Automated runtime compliance testing ==== * all running virtual machine instances are periodically tested for compliance with the defined security profile ([[Cloud_Security_Policy#Security_Profile]]) * repeated or long-running non-compliance will result in an immediate forced shutdown of the given instance ==== Automated configuration changes in virtual machines ==== * all virtual machine images must support contextualization to the following extent ** boot-time injection of a public key for the root user (where applicable) ** boot-time change of the RDP/VNC credentials (where applicable) === Post-runtime measures === ==== Extraction of virtual machine logs ==== * at the end of its lifetime (i.e. after shutdown), the contents of /var/log from the root file system of every virtual machine instance will be archived ==== Extraction of timestamps ==== * at the end of its lifetime (i.e. after shutdown), timestamps from the root file system of every virtual machine instance will be archived == Security Profile == TBD == Incident Response == * whenever possible follow general procedures stipulated by CESNET and EGI * close cooperation with CESNET security teams [[Kategorie:English]] [[Kategorie:Clouds]] at5jelpknpv21kitpid28me55xurvr9 0 515 1413 1412 2016-02-26T19:21:09Z Root 1 naimportována 1 revize wikitext text/x-wiki ([[MetaCloud_testbed_-_rychlý_průvodce|Czech version]]) __TOC__ We guide you to run a virtual machine through OpenNebula web interface. =Common terms= '''VM''' = virtual machine '''template''' a set of requirements for virtual machine. It can be: * CPU number * RAM memory size * file with disc image that defines an operating system installed in VM * requirements for IP address * whether should be console VM available in web interface by [http://cs.wikipedia.org/wiki/Virtual_Network_Computing VNC] * parameters [[Kontextualizace VM|contextualisation]], setting particular features after VM start, for example ssh keys of VM owner '''image''' - disc image, the file system with preinstalled operating system. Image may be: * '''perzistent''' – changes in image are kept also after deleting VM. One particular image can be attached to only one particular virtual machine. * '''nonperzistent''' – it may be part of several currently running VM. When new VM is created, nonpersistent image is copied to the VM. After deleting VM, the image is deleted too. But it is also possible to save an image of running VM thus save changes done in nonpersistent image. =Login in web interface= [[MetaCloud_testbed_registration|Register to the Metacentrum cloud]] and log in the [https://cloud.metacentrum.cz web interface]. OpenNebula 4.10 we interface has two types of view that differs significantly: * view '''cloud''', the more simple view to run new virtual machines * view '''user''', the more complex view with more settings and informations Default settings is '''cloud''' view. How to switch between views is described in [[Přepínání režimů webového rozhraní OpenNebuly|Switching regimes of Open Nebula]]. = Run virtual machine = == Run virtual machine in '''cloud''' view == There is an icon '''VMs''' in upper right corner and then the plus button. [[Soubor:Opennebula cloudview new vm.png|border|400px]] New page appears, choose short name of new virtual machine. A list with available templates appears, it also involves operating system icons. You may filter this list by inserting a string into search field. Click on the template. [[Soubor:Opennebula cloudview select template.png|border|400px]] You may change number of CPU or amount of RAM memory in '''Capacity'''. You may also remove some from IP addresses in '''Network'''. We recommend not to add an additional IP addresses, because an impossible combination can be created by allocating VM to physical device and IP address. Confirm new VM by pressing a button '''Create''' down on the page. [[Soubor:Opennebula cloudview create vm.png|border|400px]] New virtual machine will be in DEPLOYING state for a moment: [[Soubor:Opennebula cloudview vm deploying.png|border]] and after a while, it switches to RUNNING. The idle is caused by time necessary to copy image to physical device and operating system boot. Assigned IP address is highlighted in picture: [[Soubor:Opennebula cloudview vm ip.png|border]] Log in virtual machine [[#Přihlášení na virtuální stroj|Login]]. == Run a virtual machine in '''user''' view == * in left menu choose ''Virtual Resources'' and then ''Templates'', templates list appears * choose any template, list may be filtered by using of search box * click on '''Instantiate''' [[Soubor:Opennebula start vm.png|border|Obrázek: new virtual machine start|800px]] * choose any name in dialog window and confirm. [[Soubor:Opennebula instantiate vm.png|border|Obrázek: zadání jména virtuálního stroje|250px]] * choose ''Virtual machines'' in left menu, click on the VM in list, VM detail with bookmark ''Info'' appears, wait for a while, as soon as VM will be in '''RUNNING''' state, iddle is neccesary due to copy disc image to physical device and operating system boot. [[Soubor:Opennebula vm info.png|border|Obrázek: stav VM|800px]] * switch to the bookmark ''Network'' and note assigned public IP address. [[Soubor:Opennebula IP.png|border|Obrázek: zjištění IP adresy virtuálního stroje|800px]] =Login to virtual machine= Prepared images by our side downloads updates and reboots, so wait for a while until virtual machine gets to the state RUNNING. == By ssh == To login to running virtual machine with OS Linux as a user ''root'' by ssh (your ssh key is in your application form and is imported by [[Kontextualizace VM|kontextualization]], ssh root@ip_adresa where ip_address is your virtual machine public IP address. You may use to remote access to the virtual machine program [http://www.chiark.greenend.org.uk/~sgtatham/putty/ Putty], if you use Windows operating system in the image. If everything is properly set, you can login to the machine as a root, this machine works as "ordinary" remote machine. In case of problems, you may ask our [[Uživatelská_podpora| user support.]] [[Soubor:Opennebula ssh.png|border]] == By VNC == If VM template supported VNC usage, you may use graphical interface of operating system in virtual machine. Linux shows the CLI only, whereas Windows is more usable for users. After running VM, button VNC will be active. In view '''cloud''' upper left: [[Soubor:Opennebula cloudview vnc button.png|border|400px]] In view '''user''' upper right: [[Soubor:Opennebula userview vnc button.png|border|400px]] after a clicking on VNC button, VNC client GUI appears in a browser: [[Soubor:Opennebula vnc windows.png|border|800px]] You can not use other VNC client and connect directly, because VNC server do not listen on public IP address. But programs as ''Remote desktop'' or ''TeamViewer'' may be used. =Virtual machine environment= You can operate virtual machines with OS Linux by commands typed in CLI. You may get more familiar with Linux commands by getting through '''[http://www.ee.surrey.ac.uk/Teaching/Unix/ tutorials]'''. Your Virtual Machine is fully under your control, our administrators have no access to it. You are fully responsible for it. {{Warning|Keep software up to date in running virtual machine!}} If you no longer use the virtual machine, turn it off (Undeploy) or delete it (Shutdown). = Virtual machine administrating and states = == In view '''cloud''' == Administration is really simple, you may reboot, stop or delete the virtual machine by three buttons (see the Figure): [[Soubor:Opennebula cloudview vm detail.png|border|400px]] Every time, you do not need a virtual machine, shut it down or undeploy. The virtual machine that is only stopped, blocks the physical machines where they are allocated. == In view '''user'''== There is more options to administer the virtual machine than in view cloud. === Virtual machine stop === There is a button in the virtual machine detail to change the state: [[Soubor:Opennebula ovladani vm.png|border]] Full description is in [http://archives.opennebula.org/documentation:rel4.4:vm_guide_2#terminating_vm_instances dokumentaci OpenNebuly], the summary: * '''Suspend''' - suspend to disc (hibernation), similarly when your notebook is "sleeping", VM stays in the physical device * '''Power Off''' - sent ACPI signal to OS, similarly as you turn off our notebook, VM stays in the physical device * '''Power Off hard''' - turn off OS immediately, similarly as when you plug out accidentally power supply from your PC. VM stays in physical device. * '''Stop''' - suspend to disk (hibernation) and virtual machine is moved from physical device to datastore * '''Undeploy''' - sent ACPI signal to OS, turn it off and move physical device to datastore * '''Undeploy hard''' - turn off OS immediately and move VM from physical device to datastore '''Suspend''' and '''Stop''' hibernate, after switching to state RUNNING, programs continue in running. '''Power Off''' and '''Undeploy''' turn off OS, so virtual machine boots after changing state to RUNNING. There is no reason to keep virtual machine in power off state, it blocks dedicated memory, CPUs and disc space. Please, use state '''Undeploy''' instead. All states above keep disc state also in nonpersistent images, VM is not deleted, only suspended. Similarly, IP address keeps assigned. Please, delete all idle virtual machines to relase IP addresses. === Virtual machine deleting === [[Soubor:Opennebula delete vm.png|border]] * '''Shutdown''' - send ACPI signal to OS, wait until OS is terminated, copy persistent disc images, deletes nonpersistent disc images, deletes virtual machine * '''Shutdown hard''' - similarly as Shutdown, but end OS immediately (similarly as plug out PC) * '''Delete''' - deletes immediately virtual machine, do not copy persistent discs! Standard approach for deleting virtual machine with operating system with ACPI support (all prepared images) is command '''Shutdown'''. After deleting a virtual machine, nonpersistent disc is deleted! =What next?= [[MetaCloud_testbed_-_příklad_práce_s_cloudem|The most common use cases with cloud]]. You may also see [[Správa Metacloudu přes webové rozhraní|detailed howto]]. [[Kategorie:English]] [[Kategorie:Clouds]] qncj2qnr53ff9g0r6or20po01p1toc1 0 516 1415 1414 2016-02-26T19:21:09Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Category:Clouds]] [[Cloud_v_MetaCentru|Czech version]] MetaCentrum Cloud or simply „'''MetaCloud'''“ - is [https://en.wikipedia.org/wiki/Cloud_computing#Infrastructure_as_a_service_.28IaaS.29] ("Infrastructure as a Service") cloud administered by [http://opennebula.org/ OpenNebula] software. MetaCloud can utilize all MetaCentrum or e-Infrastructure CESNET users, so all academic and employees and students and also The Czech academy of science employees. The OpenNebula web interface is '''https://cloud.metacentrum.cz/'''. == Hardware == MetaCentrum run the cloud in a cooperation with [http://www.cerit-sc.cz/ CERIT-SC] Masaryk University in Brno. Available physical devices are listed in http://metavo.metacentrum.cz/pbsmon2/cloud/ and covers: * CERIT-SC clusters ** '''hda''' - situated in Brno, 48 machines with 12 CPU, 90 GB RAM, 2x 600GB HDD ** '''hdb''' - situated in Jihlava, 32 machines with 8 CPU, 128 GB RAM, 2x 600 GB HDD ** '''hdc''' - situated in Jihlava, 112 machines with 16 CPU, 128 GB RAM, 2x 600 GB HDD * CESNET clusters ** '''dukan''' - situated in Brno, 10 machines with 12 CPU, 96 GB RAM, 2x 600 GB HDD ** '''warg''' - situated in Plzeň, 13 machines with 16 CPU, 64 GB RAM, 4x 900 GB HDD All zegox, zigur a zapat cluster nodes are actually virtual machines maintained by PBS scheduler and allocated on physical clusters hda, hdb a hdc and their size may change in time when neccessary. Similarly as Amazon EC2 cloud „availability zones“, also MetaCloud is divided into parts, so called "clusters" in OpenNebula. These parts behave as separate sections, and only one cluster can be choosen during image loading or network addresses specification. Ordinary users do not see these clusters in OpenNebula web interface, but it can be distinguished according to "datastore" where are disc images saved. Clusters also influence physical device, where virtual machine will run. For example, if you upload your own disc image to datastore ''cerit-sc-jihlava'', virtual machines may physically run in Jihlava only. == Comparison MetaCloudu and other clouds == MetaCloud is an [http://en.wikipedia.org/wiki/Infrastructure_as_a_service#Infrastructure IaaS] (Infrastructure as a Service) cloud built using [http://opennebula.org/ OpenNebula v4.10]. Pilot cloud installation provided by CESNET and [http://www.cerit-sc.cz/en/index.html CERIT-SC]. MetaCloud is similar to e.g. Amazon Elastic Compute Cloud (EC2), Microsoft Azure or Google Compute Engine. When familiar with Amazon EC2 or Google Compute Engine, following table can help you in terminology: {| class="wikitable" |+ clouds compared |- ! !! MetaCloud !! Amazon EC2 !! Google Compute Engine |- | web interface URL|| https://cloud.metacentrum.cz/|| https://console.aws.amazon.com/ec2/|| https://console.developers.google.com/ |- | hypervisors|| KVM|| Xen|| KVM |- | provided disk images|| Debian, CentOS, SciLinux, MS Windows|| Amazon Linux, Ubuntu, RedHat, Suse, MS Windows|| Debian, RHEL, CentOS, SLES |- | disk image upload formats|| qcow2, vmdk, vdi, raw|| raw|| disk.raw in tar.gz |- | term for VM|| VM|| instance|| instance |- | term for VM spec (CPUs,RAM..)|| template|| AIM + instance type|| image + machine type |- | terms for VM suspend, stop and delete|| suspend, stop, undeploy|| -, stop, terminate|| -, -, delete |- | file storage service|| Cumulus (S3 compatible)|| S3|| Google Cloud Storage |- | persistent disk storage|| persistent disk images, copy on start and stop|| Elastic Block Store, mounted over network|| persistent disk images |} == Comparison of MetaCloud and MetaCentrum == {| class="wikitable" |- ! MetaCentrum (scheduler PBS) !! MetaCloud |- |Machines are administered by MetaCentrum administrators. |Machines are created, run and administered by users. |- |MetaCetrum administrators are responsible for security of machines. |Users are responsible for virtual machines security, especially for regularly software updates. |- |A user has rights as ordinary Unix user in all MetaCentrum machines. |A user has <tt>root</tt> rights in his virtual machine. |- |Applications are installed by MetaCentrum employees, users can install applications only in their /home directory. |Applications are installed by a user only in his virtual machine. |- |Jobs scheduled by PBS scheduler. |No jobs at all, a user can run virtual machines and applications on them any time he wants. |- |Only particular combinations of CPU number and memory size are possible depending on hardware configuration. |A user creates virtual machine exactly as he needs. |- |After script termination, data must be moved from /scratch to /home directory in the front end. |There are many possibilities to save data in cloud, for example, use persistent image. |} [[Interaktivní úloha|Interactive jobs]] in MetaCentrum may be slightly comparable to use of MetaCloud, but you can not influence time in PBS scheduler when machines will be available to you. Whereas PBS scheduler schedule jobs to the MetaCentrum machines, users run their own virtual machines in MetaCloud and they are fully responsible for their conditions. They can also freely choose operating system and install applications. 6f4bvd8h5ybfkp1z9m8bmfejgmo3bua 0 464 2068 1579 2016-05-10T07:23:34Z Root 1 wikitext text/x-wiki This page provides brief overview of computational services offered by MetaCentrum. =Grid services= [https://cs.wikipedia.org/wiki/Grid_computing Grid] is the collection of computer resources from multiple locations to reach a common goal. *Benefits for users **Top level of computational capacity **Higher accessibility and reliability of saved data **Data sharing from multiple sources If you are interested in this service, continue [[How to compute|here]]. =Cloud services= [https://en.wikipedia.org/wiki/Cloud_computing Cloud computing] is a method of sharing resources via the Internet. Benefits for users: * You manage your own machine * You can configure your machine as you like (OS, RAM memory, number of CPU...) * You are the root, you can install applications on your own If you are interested in this service, continue [[:Category:Clouds|here]]. =Hadoop= Hadoop is a popular open-source framework for distributed storage and distributed processing of large volumes of data, especially where the processing involves a MapReduce algorithm. Unlike traditional cluster resources in MetaCentrum, which assume just-in-time stage-in of data before processing, the Hadoop environment is intended for mid- to long-term storage/collection of data to be processed. Aside of a large native Hadoop cluster extensible on demand with additional virtual nodes, MetaCentrum can also offer purely virtual Hadoop PaaS clusters for prototyping your solutions prior to large-scale deployment. If you are interested in this service, continue [[:Category:Hadoop|here]]. [[Category:Grid services]] [[Category:Clouds]] [[Category:Hadoop]] t6m95540jkprr1pxa30t2zfmt3dn4w8 0 493 4723 2115 2016-05-13T07:20:59Z Root 1 wikitext text/x-wiki ([[Autentizační systém Kerberos|Česká verze]]) Kerberos provides sophisticated mechanisms for authentication and security of network communication. System Kerberos is defined by standard [http://www.ietf.org/rfc/rfc1510.txt IETF RFC 1510] and creates basic authentication element in many commercial and open-source systems. == Description == Kerberos is designed in such a way that ensures strong security together with easy user interface. The way of authentication, i.e. proving of identity is based in system Kerberos on utilization of so-called ''tickets'' issued by central authentication server that manages the database of all users. Tickets are analogy to e.g. certificates of public keys that are well known from [https://en.wikipedia.org/wiki/Public_key_infrastructure PKI] environment. Contrary to certificates Kerberos tickets and all Kerberos protocols are exclusively based on usage of symmetric cryptography, i.e. password shared between the user and the central authentication server. Kerberos tickets have also shorter time of validity (usually ten hours) and always contain information for which end service their are designated. Besides support of authentication the mechanism of Kerberos tickets provides also support of so-called [https://en.wikipedia.org/wiki/Single_sign-on single sign-on] principle that allows users comfortable usage of resources without needless load by difficult security procedures. The user authenticates himself/herself towards Kerberos server only once and obtains basic ticket (so-called '' Ticket Granting Ticket (TGT)'') that will be used for obtaining of further tickets to access system services requiring authentication. During remote login to other machines the TGT ticket is also transfered and the user has therefore still a possibility how to authenticate himself/herself. However, all operations except the primary authentication are done transparently without any user intervention. Using this approach the user is not hindered in his/her activity. == Availability == System Kerberos is available as [[Application modules|module]] heimdal whose load is also included in module metabase. This module is automatically loaded for an user after login to MetaCentrum machines. = Utilization = {{Box|Related topics {{BoxPoložka|[[How to compute/Accessing machines/From Linux|How to access MetaCentrum resources from Linux]]|}} {{BoxPoložka|[[How to compute/Accessing machines/From Windows|How to access MetaCentrum resources from Windows]]|}} }} The program <tt>kinit</tt> is intended for obtaining TGT ticket, i.e. execution of the initial authentication. After command execution the user has to entered his/her Kerberos password. Besides this, all MetaCentrum applications that allows authetication by password are configured to verify the password towards Kerberos server. Therefore if users are using e.g. ssh protocol and use Kerberos password for the authentication they do not need to run kint manually again. Obtained tickets are stored in a file written to a disc so all applications of a specific user can use them. The list of actual tickets on the disc can be printed out using command <tt>klist</tt>, the printed information contains also information about tickets validity time. After finishing work we recommend to delete from the disc the tickets obtained during your work. Doing so you will eliminate a possibility (although very non-probable) of their misusage. The command <tt>kdestroy</tt> serves for erasing of tickets and the file where the tickets were saved. If you wish to change your Kerberos password enter [http://metavo.metacentrum.cz/en/myaccount/heslo.html My Account - Change password] and follow the instruction here. Kerberos mechanism is supported by a set of applications. Clients as telnet, possibly ssh that support authentication and subsequent encryption of transfering channel can be used for accessing MetaCentrum machines. Ftp, scp, possibly rcp that are parts of standard MetaCentrum software equipment can be used for file transfer. All of these applications also transfer user's tickets that allows comfortable traversing among more machines without necessity to repeatedly enter password. == Licence == Freely available software ([http://en.wikipedia.org/wiki/Bsd_licence BSD licence]). == Supported platforms == UNIX systems and Microsoft Windows. [[Category:Grid services]] [[Category:Clouds]] [[Category:Infrastructure]] rccuw27asgz1eh8gxn38cjbl1sjrlcu 0 477 5264 5258 2016-05-14T08:07:07Z Root 1 /* Data storage */ wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[Types of data storages]]|}} {{BoxPoložka|[[Archival data handling]]|}} }} This topical guide provides the most important information about ''data manipulation'' in MetaCentrum. You might be interested in these topics: ===Data transfer=== *[[Working with data/Data transfer|Data transfer]] &ndash; Basic data transfer tutorial *[[Working with data/Direct access to data storages|Direct access to data storages]] &ndash; Advanced data transfer tutorial ===Data and jobs=== *[[Working with data/Working with data in a job|Working with data in a job]] &ndash; How to work with data in your computations ===Data storages=== *[[Types of data storages]] &ndash; Basic overview of data storages offered by MetaCentrum *[[Archival data handling]] &ndash; Tutorial about archiving data [[Category:Working with data]] 4h4zj7b1zgwc86xm805q7u51p50ulew 0 520 3326 1829 2016-05-13T07:06:20Z Root 1 wikitext text/x-wiki Bitvise is flexible SSH Client for Windows which includes state of the art terminal emulation, graphical as well as command-line SFTP support, an FTP-to-SFTP and more. ==Getting Bitvise SSH Client== Download the latest release from [https://www.bitvise.com/ssh-client-download the official page] and install it. ==Configuring Bitvise SSH Client== Open the program and follow these steps: Step '''1''': Fill in target frontend address (list of frontends is avalaible [[Frontend|here]]) Step '''2''': Fill in your MetaCentrum username. Step '''3''': Select password as the initial method, check in store encrypted password in profile and fill in your MetaCentrum password. [[Image:Bitvise1.png]] Click '''Login''', two windows similar to these ones should appear: [[Image:Bitvise2.png]] One for data transfer and the other for terminal. [[Category:Working with data]] 0jurfjdikfv7j8h8vj623qrekmy8uau 0 535 5237 2111 2016-05-13T14:17:15Z Root 1 /* Scratch types */ wikitext text/x-wiki ([[Úložiště scratch]]) ''Scratch storage'' is a storage for temporary files on computing nodes. This storage should be used only during computations and should be freed after your job ends immediately. ==Scratch types== We offer three types of the scratch storage * ultra fast but small SSD disc, mounted to directory '''/scratch.ssd''' * fast regular local disk, mounted to directory '''/scratch''' * network volume available for all the nodes of a specific cluster, mounted to directory '''/scratch.shared''' You can set the size and type of scratch dir during [[How to compute/Requesting resources|job submission]] by use the parameter -l scratch=number[kb|mb|gb]:type '''''number''''' is a size requested to be reserved '''''type''''' may take one of the following values (by default, the following mechanism for allocating scratch space is applied: ''SSD'' (if exists) -> ''shared'' (if exists) -> ''local''): :* '''local''' ... a request to allocate a scratch space located on a fast HDD; the scratch will be ''local'' for all the nodes :* '''shared''' ... a request to allocate a scratch space located on a shared network volume; the scratch will be ''shared'' for all the nodes of a specific cluster :* '''ssd''' ... a request to allocate a scratch space located on a ultra fast SSD; the scratch will be ''local'' for all the nodes :* '''first''' ... a request to allocate a scratch space located on a fast HDD; when asking for multiple nodes, the scratch will be allocated ''just'' on the master node. (This type may be useful for the parallel jobs which do not require the scratch space allocated on all the worker nodes.) {{RedInfo|Please note, that by explicit request for the ultra fast (''ssd'') or shared (''shared'') scratch space one may significantly decrease the number of nodes, which could be assigned to such a job (since most of the MetaCentrum clusters are not equipped with these scratch types -- see the list of MetaCentrum HW located at [http://metavo.metacentrum.cz/pbsmon2/hardware MetaCentrum webpage]).}} ====Examples:==== * '''-l scratch=1''' &ndash; job requires 1 [http://en.wikipedia.org/wiki/Kibibyte KiB] of scratch disk space * '''-l scratch=1mb''' &ndash; job requires 1 [http://en.wikipedia.org/wiki/Mebibyte MiB] of scratch disk space * '''-l scratch=1gb''' &ndash; job requires 1 [http://en.wikipedia.org/wiki/Gibibyte GiB] of scratch disk space * '''-l scratch=10gb:ssd''' &ndash; job requires 10 GiB of scratch disk space on SSD disk * '''-l scratch=20gb:local''' &ndash; job requires 20 GiB of scratch disk space on local HDD * '''-l scratch=100gb:shared''' &ndash; job requires 100 GiB of shared scratch disk space on network volume * '''-l scratch=500gb:shared''' &ndash; job requires 500 GiB of scratch disk space allocated just on the master node (the other nodes won't have any scratch allocated) == System variables == Allocated space on scratch disk is mentioned in system variable '''SCRATCHDIR''', which value is like <tt>/directory/username/job_jobid</tt>, ie. <tt>/scratch.ssd/username/job_2477934.arien.ics.muni.cz</tt>. Scratch type is mentioned in system variable '''TORQUE_RESC_SCRATCH_TYPE'''. Note: To see the system variable type echo $SCRATCHDIR [[Category:Infrastructure]] [[Category:Working with data]] 3qg0uqrnd9fa92hhjlkkrireex7td0u 0 541 1968 1967 2016-05-03T16:34:19Z Root 1 /* Data preparation */ wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[How to compute]]|}} {{BoxPoložka|[[Types of data storages]]|}} {{BoxPoložka|[[Scratch storage#Types of scratch|Types of scratch storage]]|}} }} This page provide information about preparing and using data within a job. If you would like to learn more about MetaCentrum file systems, read the [[Types of data storages]] page. ==Data preparation== '''1.''' Log on any available [[Frontend|frontend]]. '''2.''' Select one the options below: * If you need to prepare reasonable amount of data (< 5GB), read [[Working with data/Data transfer|Data transfer]] * If you need to prepare '''huge amount of data''', read [[Working with data/Direct access to data storages|Direct access to data storages]] ==Data and jobs== Before you begin, please keep in mind these rules: * [[Scratch storage]] is the best storage type for data in a running job * [[Types of data storages#Disk arrays|/storage volumes]] are best storage types for keeping and saving your data between running jobs * Archive data are handled by [https://du.cesnet.cz/en/start CESNET Data Department], see [[Backup and archiving]] for more information * Current directory is saved into '''PBS_O_WORKDIR''' variable when you submit your job via qsub command. You can find [[How to compute/Job management#Examine job's standard output and standard error output|job's outputs]] there. * It is always better to use the '''absolute path''' rather than working directory ===One processor job example=== This example shows how to set up your job to work with scratch directory. '''1.''' Prepare your input data, e.g. to ''/storage/brno2/home/$LOGNAME/myJob'' directory '''2.''' Prepare your batch script {{YellowInfo|This example deletes data of unsuccessful jobs. Add the following code behind <code>trap 'clean_scratch' TERM EXIT</code> line, if you would like to keep the data ''':''' <source lang=bash> trap 'cp -r $SCRATCHDIR/run.log $SCRATCHDIR/uncomplete.data $DATADIR && clean_scratch' TERM </source>}} Create myJob.sh text file in your favourite text editor with following lines: <source lang="bash"> #!/bin/bash #PBS -N oneCPUjob #PBS -l walltime=1h #PBS -l nodes=1:ppn=1 #PBS -l mem=10gb #PBS -l scratch=1gb #PBS -j oe #PBS -m e # Options for scheduling system - job will last 1 hour at maximum, 1 machine with 1 processor + 10gb memory RAM + 1gb scratch storage is requested # Error output and standard output will be merged, a notification email will be sent when the job is finished # You can adjust these options by your needs trap 'clean_scratch' TERM EXIT # sets up scratch cleaning in case an error occurs module add target_application # loads desired application DATADIR="/storage/brno2/home/$LOGNAME/myJob/" # sets up your input data directory if [ ! -d "$SCRATCHDIR" ] ; then echo "Scratch directory is not created!" 1>&2; exit 1; fi #checks if scratch directory is created cp $DATADIR/data $SCRATCHDIR/ # copies data from the input data directory to your scratch directory cd $SCRATCHDIR # moves to your scratch diretory #Space for your computation: #----- #----- echo "uloha $PBS_JOBID na `uname -n`" >result # just an example computation cp result $DATADIR || export CLEAN_SCRATCH=false # copies result to your input data directory. The result will not be removed if the copying fails. </source> '''3.''' Submit the job <source lang="bash"> qsub myJob.sh </source> You can find your result in /storage/brno2/home/$LOGNAME/myJob directory when the job is finished. ===Multiprocessor job example=== dodělat [[Category:Working with data]] e6h9yf5zco36n2jd3o4mtldp4x16pw5 0 542 2064 1959 2016-05-09T15:23:46Z Root 1 /* /storage volumes */ wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[Working with data]]|}} }} Deciding what type of storage to use for your data affects how efficiently you are able to manage your job output. Criteria you should consider include * how much data you need to store * how long you intend to store the data * how easily you want to be able to access data * how quickly you need to transfer data == File systems available == [[Soubor:Filesystems.png]] The disk space in MetaCentrum must cater for various needs, so there are several types of if: All the computing nodes provide three types of storage volumes, which are available for users: * /scratch - fast filesystem, dedicated for applications' temporal/working data. This volume is local on every computing node, provides a basic protection against HW failures (RAID 10) and is not backed up. * /home - shared filesystem dedicated for users' homes. * /storage - MetaCentrum's shared and backed-up NFSv4 volumes (available from the frontend as well as all the computing nodes) ===Temporary scratch storage (for temporary data)=== The scratch volumes are very fast filesystems intended to be used as a temporary storage for running jobs. The filesystems are located either on a local discs of a node (data cannot be shared among nodes) or on a fast network storage (data can be shared among the nodes of a cluster) -- the <code>$SCRATCHDIR</code> environment variable always points to a scratch space dedicated to a job. These volumes should be used only by currently running jobs, so any data must be removed immediately after a job finishes. ===/home directories on local cluster=== Home directories are permanent, but have relatively small quotas. This space is visible by all the nodes in the cluster, including the frontends. Your home directory is best for compiling and building programs and for storing output that consists of large numbers of small files. This space is usually not visible on other clusters, the only exception is clusters located in Brno, Jihlava, and Pilsen, which share one filesystem. ===Shared network file system AFS on all machines=== All machines in MetaCentrum have AFS mounted. The system is secure and scalable to the whoel world, but it is rather slow. You can mount the AFS file system on your own workstation if you want. ===/storage volumes=== /storage volumes are built on the NFSv4 network filesystem. Physical storages are disk arrays situated in all cities where are clusters connected to MetaCentrum. These volumes serves for storing users data mainly to folders '''/storage/'''''city'''''/home/'''''user''. They are available at machines with "nfs4" property. [[Kerberos authentication system]] is used to manage the user access and you can [[Mounting data storages on local station|mount this filesystem]] from every local machine. Unfortunatelly the speed of data writing operation is influenced by the length of data network. Therefore writing to /storage/brno{1.2.3} located in Brno will be the fastest from the Brno's clusters then from Prague clusters and the slowest from Plzen clusters. The influence is worse while writing small files than for larger ones. The reading speed is not much affected. {{:NFS4 Servers}} {{GreenBox|The often user problem is the inability to access the folders in /storage while they are logged in to [[frontend]] by the ssh key. In this case they don't have generated the Kerberos tickets. This can be repaired by invocation of <tt>kinit</tt> command.}} == Access rights == All volumes are covered by standard set of [http://en.wikipedia.org/wiki/Filesystem_permissions unix permissions]. Default set up of directories (including the home directory) and files is '''755''', ie. '''rwxr-xr-x'''. In case you want to change this behavior you can do it by invoking command chmod 700 <directory> To change the rights for your home directory and for computing/temporal directory (scratch) use chmod 700 /home/$USER chmod 700 $SCRATCHDIR Changing the rights can be performed also from user scripts if you compute some sensitive data. On the other hand you can use current properties allowing read and execute to share your scripts in groups of users. == Quotas == Quotas are the mechanism for monitoring of disk storage capacity for administrators and for the error prevention. This error could negatively affect the work of all users because of possible fulfilment. You can check your current quotas at your [http://metavo.metacentrum.cz/en/myaccount/quota personal page]. [[Category:Working with data]] [[Category:Infrastructure]] 7txekxrj8dkc9hx7t9uhvofr86g5r4e 0 550 1963 1962 2016-05-03T15:44:19Z Root 1 /* Scratch storages */ wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[Working with data]]|}} {{BoxPoložka|[[File systems in MetaCentrum]]|}} }} This page provides information about data storages offered by MetaCentrum, see [[File systems in MetaCentrum]] for advanced information about this topic. ==Basic data storages overview== {| class="tabulka" ! style="background-color:LightSeaGreen;"|Storage type !! style="background-color:LightSeaGreen;"|Basic description !! style="background-color:LightSeaGreen;"|Typical usage !! style="background-color:LightSeaGreen;"|Additional information |- | '''Scratch storages''' || Fast storages with minimum data capacity ||Working with data during computations || [[Scratch storage]] |- | '''Disk arrays''' || /storage volumes in MetaCentrum || Data storing between compututions || [[Types of data storages#Data storages description#Disk arrays|Disk arrays]] |- | '''Hierarchical storages''' || Storages with massive data capacity || Data archiving || [https://du.cesnet.cz/en/start CESNET data care] |- |} ==Data storages== There are three data storages types offered by MetaCentrum: ===Scratch storages=== Scratch storages are accessible via ''scratch directory'' on computational nodes. You should '''use this storages during your computations only'''. Data on scratch storages are automatically deleted after 14 days. * Read [[Scratch storage]] for advanced information ===Disk arrays=== Disk arrays contains many connected disks and files on them are stored on multiple disks, which guarantees higher I/O data speed and reliability. '''Use disk arrays for preparing data and storing data between jobs'''. * Disk arrays are accessible via ''/storage'' directories, see [[Working with data/Direct access to data storages|Direct access to data storages]] for more infomation ===Hierarchical storages=== Hierarchical storages contains several layers of storage media - lower layers have bigger capacity but slower access times. Data are moving among these layers based on their usage during recent times. The first layer is a disk array are other layers are made of MAIDs (Massive Array of Idle Drives) or magnetic tape libraries. '''Use hierarchical storages for storing data which you do not currently use, but you may need them in the future. * Read the [https://du.cesnet.cz/en/start CESNET data care] page for more information [[Category:Working with data]] [[Category:Infrastructure]] oekt4unsrd7fvlu528maxbzrnhd27mt 0 561 1879 1877 2016-04-28T11:45:50Z Root 1 /* Configuring WinSCP */ wikitext text/x-wiki WinSCP is a SFTP client, SCP client, FTPS client and FTP client integrated into one software program for file transfer to FTP server or secure SFTP server. ==Getting WinSCP== Visit WinSCP's [https://winscp.net/eng/index.php homepage] or go directly to the [https://winscp.net/eng/download.php download page]. WinSCP is an installation-free software, just open the downloaded file. ==Configuring WinSCP== {{RedInfo|For working with huge amount of data, see the [[Working with data/Direct access to data storages|Direct access to data storages]] topic}} Step '''1''': Fill in target frontend address (list of frontends is avalaible on the [[Frontend|this]] page) Step '''2''': Fill in your MetaCentrum username. Step '''3''': Fill in your MetaCentrum password (leave empty if you are using [[How to compute/Accessing machines/From Windows#Using Kerberos|Kerberos]].) Step '''4''': Click '''Advanced...''' button -> the settings window should appear. [[Image:winscp1.png]] Step '''5''': Open the SSH tab Step '''6''': Select 2 only as preferred the SSH protocol version [[Image:winscp2.png]] Step '''7''': Open the SSH/Authentication tab Step '''8''': Check in both options [[Image:winscp3.png]] Click '''OK''', you will return to the main window where you can save these settings. [[Category:Grid services]] [[Category:Working with data]] 9xk3yvzlqxm0wl1yv4st5ih91qvsiu0 0 568 1986 1985 2016-05-04T11:19:48Z Root 1 /* Mounting the volume */ wikitext text/x-wiki MetaCentrum enables boundless data storing capacity throught NFSv4 protocol. You can reach this space in directory <tt>/storage</tt> (see [[Types of data storages ]]) at the [https://metavo.metacentrum.cz/en/myaccount/quota machines in MetaCentrum] with property <tt>'nfs4'</tt>. You can also access this space from your PC. This page contains Linux tutorial how to do it. There is no free client for MS-Windows, just comercial [http://connectivity.opentext.com/products/network-file-system.aspx NFS Maestro]. == NFSv4 directories and servers == {{:NFS4 Servers}} == What do you need == You need several things to make NFSv4 from server smaug1.ics.muni.cz accessible on your Linux desktop: * empty directory as a mount point * Kerberos support installed * NFSv4 support in kernel * kerberos ticket enabling connection * client NFSv4 tools * kerberos ticket to access to data * to be a root on your machine * set the system time properly == Directory /storage == Shared space is necessary to be map to some directory. We recommend directory <tt>/storage</tt> because in this way it is set on MetaCentrum machines. So create empty directory by command: mkdir -p /storage/brno1 mkdir -p /storage/brno2 mkdir -p /storage/brno3-cerit mkdir -p /storage/plzen1 == Kerberos support == Install Kerberos system support. Packages <tt>heimdal-clients</tt> and <tt>krb5-config</tt> in Ubuntu 8.10, packages <tt>krb5</tt> and <tt>krb5-client</tt> in OpenSuse 11.1. It is necessary to set file <tt>/etc/krb5.conf</tt> properly. The easiest way is to copy it from some MetaCentrum machine e.g.machine skirit.ics.muni.cz. $ scp skirit.ics.muni.cz:/etc/krb5.conf /etc or download from the web http://metavo.metacentrum.cz/cs/docs/krb5.conf You can get kerberos ticket throught command <tt>kinit</tt> and to print it throught command <tt>klist</tt> on condition you have proper installation of Kerberos. If you are using MIT Kerberos >1.4 or Heimdal >1.3 add following line to the [libdefaults] section in the krb5.conf: ''allow_weak_crypto = true''. == Support NFSv4 in kernel == Support is in the standard kernel in OpenSuse and Ubuntu so you don't have to test it. Otherwise assure the support in the following way: ==== Test of support NFS file system ==== * <tt>grep nfs4 /proc/filesystems</tt> Right answer: <tt>nodev nfs4</tt><br/> In case of empty answer do as a root $ modprobe nfs and repeat the test. In case of negativ answer it is necessary to compile NFS (with NFSv4 support) into kernel. ==== Test RPCSECsupport ==== * <tt>ls -d /proc/net/rpc/auth.rpcsec*</tt> Right answer: <tt>proc/net/rpc/auth.rpcsec.context /proc/net/rpc/auth.rpcsec.init</tt><br/> In case of answer: <tt>ls</tt>: cannot access /proc/net/rpc/auth.rpcsec*: No such file or directory</tt> do as root modprobe auth_rpcgss and repeat the test. In case of negative answer it is necessary to compile <tt>CONFIG_SUNRPC_GSS</tt> to Linux kernel. ==== Automatic inserting of modules ==== If it is obvious that system supports NFSv4 including RPCSEC it is not necessary to insert modules manually - client tools insert NFS on their own. == Kerberos ticket enabling volume connection == We offer 3 possibilities according type of tickets: * Your machine '''has''' file <tt>/etc/krb5.keytab</tt> from MetaCentrum (or ÚVT) *Your machine '''does not have''' file <tt>/etc/krb5.keytab</tt> from MetaCentrum **You can use your user ticket **We release one-function file <tt>/etc/krb5.keytab</tt> for you === Your machine '''has''' file <tt>/etc/krb5.keytab</tt> from MetaCentrum === We will add ticket like <tt>nfs/your_machine@ICS.MUNI.CZ</tt> to your <tt>/etc/krb5.keytab</tt> on your [[:Category:User support|request]] which enables you to connect NFS volume. Original <tt>krb5.keytab</tt> must be changed to the new one, which will be released to you on your request. === Your machine does not have file <tt>/etc/krb5.keytab</tt> from MetaCentrum === ==== You can use your own user ticket ==== Create kerberos ticket (<tt>kinit</tt>) before mounting the volume. You must create the ticket as a root because system volume will be connected as a root too. In this way, it is necessary to set <tt>rpc.gssd</tt> to search your ticket, not the system one, see bellow. Ticket must be renewed every day. ==== Create one-purpose file <tt>/etc/krb5.keytab</tt> ==== Login to some MetaCentrum machine. Be sure that you have valid Kerberos ticket in META realm (command <tt>klist</tt> will print it). After it use the command: /software/remctl-2.12/bin/remctl -d kdccesnet.ics.muni.cz accounts nfskeytab >krb5.keytab File <tt>krb5.keytab</tt> will be created in actuall directory which contains one-purpose ticket like <tt>nfs/your_login@META</tt>. Contain of this file can be printed by command <tt>ktutil</tt>: $ /usr/sbin/ktutil -k krb5.keytab list krb5.keytab: Vno Type Principal 2 des-cbc-crc nfs/makub@META 2 des-cbc-md4 nfs/makub@META 2 des-cbc-md5 nfs/makub@META Copy the file <tt>krb5.keytab</tt> on your machine to <tt>/etc/krb5.keytab</tt>. Delete the file from MetaCentrum machine after copying it. It is important to be ''META'' your implicit realm in file <tt>/etc/krb5.conf</tt>. == Client NFSv4 tools == We recommend to install <tt>nfs-utils</tt> version 1.1.0 or higher. * It is package <tt>nfs-common</tt> (''apt-get install nfs-common'') on Debian/Ubuntu. * It is package <tt>nfs-utils</tt> (''yast -i nfs-utils'') on OpenSuse lower than 10.3 * It is package <tt>nfs-client</tt> (''yast -i nfs-client'') on OpenSuse 10.3 and higher. You will also need running <tt>portmap</tt>. It should be installed with <tt>nfs-utils</tt> dependencies. Otherwise it is package with the same name (<tt>portmap</tt>). === Setting of <tt>nfs-utils for NFSv4</tt> === * '''Debian/Ubuntu''' Setting of <tt>nfs-utils</tt> is in the file <tt>/etc/default/nfs-common</tt>. We set the values in the following way: NEED_STATD=yes STATDOPTS= NEED_IDMAPD=yes NEED_GSSD=yes * '''Opensuse''' Setting of <tt>nfs-utils</tt> in the file <tt>/etc/sysconfig/nfs</tt>. We set at least following values: NFS_SECURITY_GSS="yes" NFS4_SUPPORT="yes" === /etc/fstab setting=== Add the following lines to the <tt>/etc/fstab</tt> file storage-brno1.metacentrum.cz:/ /storage/brno1 nfs4 sec=krb5 0 0 storage-brno2.metacentrum.cz:/ /storage/brno2 nfs4 sec=krb5 0 0 storage-brno3-cerit.metacentrum.cz/ /storage/brno3-cerit nfs4 sec=krb5 0 0 storage-plzen1.metacentrum.cz:/ /storage/plzen1 nfs4 sec=krb5 0 0 Then start <code>nfs</code> and set auto start after reboot: ==== OpenSuse ==== /etc/init.d/nfs start insserv /etc/init.d/nfs ==== Debian ==== /etc/init.d/nfs-common start ==== Ubuntu ==== In file <tt>/etc/default/nfs-common</tt> should be set the option '''NEED_GSSD=yes'''. service rpcbind start service nfs-common start service gssd start mount -a For system older than 14.04: service portmap start service gssd start mount -a == Simple settings of <tt>idmapd.conf</tt> == File <tt>/etc/idmapd.conf</tt> sets mapping of NFSv4 identities to local users (NFSv4 works with text principals of kerberos, POSIX interface of file system works with numeral representation of users and groups). Simple settings <tt>/etc/idmapd.conf</tt> consist on setting of configuration line: <tt>Domain = META</tt>. The users from domain @META will be map throught files <tt>/etc/passwd</tt> and <tt>/etc/group</tt>. So it means that for identity <tt>xhejtman@META</tt> must exist record in <tt>/etc/passwd</tt> with name <tt>xhejtman</tt>. Ordinary tools (<tt>ls -l</tt>) will show the names properly if the name in given files will be exist for their principal. For nonexisting name will be user mapping choosen as nobody and nogroup. Example: grep xhejtman /etc/passwd xhejtman:x:1000:1000:Lukas Hejtmanek,,,:/home/xhejtman:/bin/bash ls -l /mnt/nfs/software total 0 drwxr-xr-x 4 nobody nogroup 51 2008-06-12 12:49 etics -rw-r--r-- 1 nobody nogroup 0 2008-06-06 14:26 hu drwxr-xr-x 6 xhejtman soft-nfs4 54 2008-06-12 14:45 libnfsidmap drwxr-xr-x 5 xhejtman soft-nfs4 40 2008-06-11 13:12 nsswitch Particular record for user <tt>xhejtman@META</tt> exists in <tt>/etc/passwd</tt> and record for group <tt>soft-nfs4@META</tt> also exists in <tt>/etc/group</tt>, so mapping runs properly and it is also properly shown. There is no record for users who own directory <tt>etics</tt> that is why it's shown as <tt>nobody:nogroup</tt>. == Mounting the volume == You don't need to mount a volume in OpenSuse 11.1, because running nfs service connect it automatically according records in <tt>/etc/fstab</tt>. Mounting the volume is necessary to do as a root. Be sure that you have proper <tt>/etc/krb5.keytab</tt> or user ticket created as a root and be sure that <tt>portmap</tt> and <tt>nfs-common</tt> (<tt>nfs-utils</tt>) are running. You can assure yourself throught commands ps ax | grep rpc.gssd, ps ax | grep portmap. We make connection throught: mount -t nfs4 storage-brno1.metacentrum.cz:/ /storage/brno1 -o sec=krb5 mount -t nfs4 storage-brno2.metacentrum.cz:/ /storage/brno2 -o sec=krb5 mount -t nfs4 storage-plzen1.metacentrum.cz:/ /storage/plzen1 -o sec=krb5 Names of NFSv4 servers are described above in this page. You can insert any other local directory where we want to mount NFS volume instead of <tt>/storage</tt>. If you use choice <tt>sec=krb5i</tt> instead of <tt>sec=krb5</tt>, data integrity will be check during transfer. If you use choice <tt>sec=krb5i</tt> data will be transfered in code. == Utilization == If you use the tutorial you should have the volume mounted now. You have to access it form '''user account''' (connection was made as a root) with valid keberos ticket to see all the files from it. You get valid kerberos ticket by command <tt>kinit</tt>. So: $ kinit makub@META's Password: $ cd /storage/brno2/home In case of problems, please write to our [[:Category:User support|user support]] === Kerberos identity === Users using one identity should be satisfied with standart NFS tools in their Linux distribution. If you want to use more identities at one time like <tt>xhejtman@META</tt>, <tt>xhejtman@ADMIN.META</tt> and <tt>xhejtman@ICS.MUNI.CZ</tt> it's necessary to use patched <tt>rpc.gssd</tt> program. Patched program is in NFS utils version 1.1.3. Patching older version is not easy, best way is to contact us and we make patched package for you. === Kerberos tickets === Ticket in <tt>/etc/krb5.keytab</tt> is implicitly used just to mount the volume. But we can make a deal that the ticket can be also used to access the storage &ndash; but explicit arrangement is necessary. This ticket is valid until kerberos server administrator deletes it. So the access via ticket can be used almost forever. User ticket (ussually in <tt>/tmp/krb5cc_number</tt>) can be used for mounting (if we run <tt>rpc.gssd</tt> with choice -n) and also for access. This ticket often has limited validity. == Settings of proper displaing of users and groups names == Above-mentioned settings of [[#Simple settings of idmapd.conf|<tt>idmapd.conf</tt>]] will display properly only users and groups stored in <tt>/etc/passwd</tt> and <tt>/etc/group</tt>. Moreover the user or group must be from <tt>META</tt> domain. Cross realm user mapping is possible through advanced settings. It is necessary to set mapping of NFSv4 identities to numeral representation and it is necessary to set mapping of numerical representation to the individual names. === Settings of cross realm mapping NFSv4 identities to numerical representation === It is necessary to reach mapping files of users and groups for mapping support. * [http://metavo.metacentrum.cz/cs/docs/storage/nfsv4/passwd-nfs4 http://meta.cesnet.cz/nfs4/passwd-nfs4] * [http://metavo.metacentrum.cz/cs/docs/storage/nfsv4/group-nfs4 http://meta.cesnet.cz/nfs4/group-nfs4] You have to save this files to <tt>/etc/passwd-nfs4</tt> and <tt>/etc/group-nfs4</tt>. Then you have to download and install new version of library <tt>libnfsidmap</tt> from NFS. It is placed in <tt>/storage/software/libnfsidmap</tt>, where are both libraries for IA32 and X86_64 (<tt>lib32</tt> and <tt>lib64</tt>), and Debian packages (for i386 a ndAMD64). For this way of mapping it is necessary to set simply way, desribed above, get a packages and then you can try this advanced way. We should change settings of <tt>idmapd.conf</tt> after installing new version and reaching mapping files. : [Translation] Method = mnsswitch Similar settings could have been in configuration file. New settings have text: <tt>'''m'''nsswitch</tt> instead <tt>nsswitch</tt>. We keep settings <tt>Domain = META</tt> the same. Restart service <tt>idmapd</tt>. Debian: <tt>/etc/init.d/nfs-common restart</tt>, opensuse <tt>/etc/init.d/idmapd restart</tt> or <tt>/etc/init.d/nfs restart</tt> So now we have cross realm mapping of identities NFSv4 to numeral representation for POSIX interface. === Settings of cross realm mapping of numeral representation to the names === It is settings of <tt>nsswitch</tt> mechanism of name translation. You should download library <tt>libnss-nfs4.so.2</tt> from NFS <tt>/storage/software/nsswitch</tt> and place it into directory <tt>/lib</tt>. There are versions for debian i386 a X86_64 at NFS. There is no packages because it is only single file. Change configuration of <tt>/etc/nsswitch.conf</tt> in following way: passwd: compat nfs4 group: compat nfs4 Add word <tt>nfs4</tt> to the end of the lines <tt>passwd</tt> and <tt>group</tt> (it is the same name as have library <tt>libnss-jméno.so.2</tt>). You don't have to restart service, new mapping should work immediatelly. Names with domains are written implicitly. If we don't want to write some domain again and again, it is possible to export environment variable <tt>NFS4DOMAIN=META</tt>. Example: <tt>export NFS4DOMAIN=META</tt>, then names from domain <tt>META</tt> will be shown without this domain in the list. == Installation way for a Gentoo == You need to install packages <tt>net-nds/portmap</tt> and <tt>net-fs/nfs-utils</tt>. Check whether <tt>nfs-utils</tt> are compilled with kerberos. Scripts for a start nfs in Gentoo are little bit odd because they have common settings of server and client. Settings of nfs-utils is in the file <tt>/etc/conf.d/nfs</tt> where you need to change just choice <tt>"OPTS_RPC_GSSD"</tt> to the value <tt>" -- -n "</tt> in case you use your own key and not machine keytab. Then assure you have downloaded modules <tt>nfs</tt> a <tt>rpcsec_gss_krb5</tt> and set them to load aftear booting the system. The scripts can't load them itself. Configure <tt>/etc/idmapd.conf</tt>, as is described upper, simply just rewrite <tt>Domain</tt> to <tt>META</tt> in pattern file and return to this place in the tutorial. You need run services <tt>rpc.gssd</tt> and <tt>rpc.idmapd</tt>, for example <tt>/etc/init.d/rpc.gssd start</tt>, and manage to be run after start (<tt>rc-add default rpc.gssd</tt> and similary for <tt>rpc.idmapd</tt>). Add into <tt>/etc/fstab</tt> line smaug1.ics.muni.cz:/ /storage nfs4 sec=krb5 and run service <tt>/etc/init.d/nfsmount</tt>. Volume should mount now. [[Category:Working with data]] j59vdiuw34lnk9heo91hew398qqzeqo 0 571 2053 2052 2016-05-09T12:06:16Z Root 1 wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[Archival data handling]]|}} {{BoxPoložka|[https://du.cesnet.cz/en/start CESNET Storage Department]|}} }} This page provides information about data archiving in MetaCentrum, see [[Archival data handling]] for additional information. ==What is archived== Data can be endangered by various system malfunctions or accidental or intentional means, while formulating this policy following threats were considered: * failure of primary media (disks or disk arrays) or system software * uncontrolled changes or destruction of data due to failure, that is not directly related to storage device (e.g. memory errors) * destruction of data media by force majeure or external influence (e.g. fire, power surge) * inadvertent deletion or overwriting of files by user. ==Backup policy== Analysis of the threats enumerated above implicates need to ensure that data of permanent value is regularly backed up by establishing and following an appropriate system backup procedure. (It is not necessary to back easily recreatable data up.) Therefore, these data are regularly backed up in MetaCentrum: * application software installed by administrators * user home directories, in AFS and cluster frontends * system data and logy of servers and cluster frontends * other data areas according to individual needs of users and user groups (common project directories etc.). '''Un-backed up clusters:''' * Generally, cluster work nodes (the directories for temporal data storage -- <code>/scratch</code> and <code>/scratch.shared</code>) are not backed up. '''Backed up clusters:''' * Clusters AFS/afs are designed for long-term files storage. * Clusters NFSv4 /storage * Clusters /home According to current technical circumstances (load on backup storage devices and machine being backed-up) automated backup is run approximately tree times a week. Backups are retained for maximal time period allowed by backup storage capacity, for three months at least. Resilience against catastrophic events is implemented by double tape libraries which are used for replicated storage of backup data. Off-site storage of backup media in not used. Full backup on tapes is stored once a month, other backups are differential or incremental. Because files for differential and incremental backups are selected according to file timestamps, it is not guaranteed files with timestamps changed backwards (e.g. extracted from archives with original timestamps) will be copied during those backups. A full backup of a large data set may take a long time to complete. To avoid downtime, high-availability systems may instead perform the backup on a snapshot—a read-only copy of the data set frozen at a point in time—and allow applications to continue writing to their data. Therefore the high capacity disk array of the CERIT-SC center (/storage/brno3-cerit/home...) uses [http://en.wikipedia.org/wiki/Snapshot_%28computer_storage%29 snapshots] for data back-up. Each snapshot is available for 14 days. ==How to store your archival data== CESNET provides an infrastructure of hierarchical storages suitable for archiving purposes. Hierarchical data storage (HSM) in Jihlava and Plzen are directly accessible from all MetaCenter and CERIT-SC nodes. The storage is mounted in /storage/jihlava2-archive/home/ and /storage/plzen2-archive/home/. '''[[ Archival data handling | How to move your archival data:]]''' MetaCentrum users obtained a space with a standard 5TB disk quota. The quota can be increased on request. Older data is moved to tapes and MAID. The properties of the storage make it slightly differ from practices regarding MetaCentrum storage handling: * Home directory /storage/jihlava2-archive/home/<login>/ serves to store configuration files only, it has a tiny quota and mustn't be used to keep actual user data. The actual data space is linked from the home directory and can be found under /storage/jihlava2-archive/home/<login>/VO_metacentrum-tape_tape/. Migration policy in charge ensures that the data is always kept in a redundant manner. * All the volumes configured on the Jihlava storage facility are mounted in MetaCentrum (including service directories). If you keep your data in another virtual organisation space in Jihlava, you can access them from MetaCentrum nodes, too. The storage facility is suitable mainly for archive data storage, i.e., data which is not accessed on regular basis. You're kindly requested not to use it for live data, especially data actively used for computations. The storage is organised in a hierarchical manner. It means the system automatically moves less used data to slower tiers (mainly magnetic tapes and MAID). The data is still available for the user in the file system. It is necessary to keep in mind that access to data unused for a long time may be slower. There is almost no space left on Brno's disk arrays. Please consider to move your archieval data from /storage/<location>/home/ to /storage/plzen2-archive/ or /storage/jihlava2-archive/ (HSM) Moreover you get a benefit of 2 copies of your data thanks to the migration policy of the HSM. List of MetaCentrum storages: https://wiki.metacentrum.cz/wiki/NFS4_servery Actual usage of storages: http://metavo.metacentrum.cz/pbsmon2/nodes/physical The documentation of the directory structure can be found on https://du.cesnet.cz/wiki/doku.php/en/navody/home-migrace-plzen/start The complete storage facility documentation: https://du.cesnet.cz/wiki/doku.php/en/navody/start The hierarchical storage is operated by the CESNET Data storage department, http://du.cesnet.cz. User support is provided by the standard MetaCenter user support meta@cesnet.cz. [[Category:Policies]] [[Category:Working with data]] [[Category:Infrastructure]] mvk2av0p92gsr668mz4gzbkfgppd6vo 0 572 2049 2048 2016-05-09T11:58:17Z Root 1 /* Tips */ wikitext text/x-wiki '''[[Backup and archiving | Archival data]]''' is data that is valuable, but users access them only rarely. Such kind of data '''is NOT suitable to be kept on standard disk arrays''' since they consume both valuable space of the arrays and electricity. For such data, CESNET provides an infrastructure of hierarchical storages suitable for archiving purposes (details and important properties of the devices are available at [http://du.cesnet.cz http://du.cesnet.cz]), which is directly integrated into the MetaCentrum infrastructure. This how-to describes '''how to transfer your archival data to the hierarchical storages:''' == Terminology == '''''Source storage''''' -- a disk array on which you have your archival data. :* for the sake of simplicity, let the archival data reside in a folder named "<code>my-archive</code>" '''''Target storage''''' -- an archival hierarchical system. :* archival storages are mounted under the <code>/storage</code> directory (the respective subfolders contain "<code>-archive</code>" or "<code>-hsm</code>" in their names). ==List of Available Storages== {{:NFS4 Servers}} : '''''Note 1:''' To determine the quota you are provided on the particular storage, please, visit the [http://metavo.metacentrum.cz/en/myaccount/kvoty MetaCentrum portal].'' : '''''Note 2:''' On the HSM storages (with the suffix <code>-archive</code> or <code>-hsm</code>) the user quota '''is not applied''' (just a technical limitation of 5TB, involving an overloading of the HSM with a one-time data copy, is applied).'' == General Guidelines, Data Formats == In general: '''the smaller number of files in the archive, the better''' (it speeds operations up and generates lower load on the storage subsystems; on the other hand, packing the files makes searching less comfortable). =====General guidelines:===== :* if most of your ''files are large'' (hundreds of MBs, GBs, ...), don't bother with packing them and make a one-to-one copy to the archive, :* if your ''files are smaller'' and you don't plan to search individual files, pack them into tar or zip files, :* if ''packing the files is too uncomfortable'' for you, make a one-to-one copy to the archive; in this case, ''avoid using Pilsen storage (plzen2-archive)'' =====Archival destination:===== :* for ''smaller files'', '''avoid using Pilsen storage''' (plzen2-archive) -- manipulating large quantities of small files is significantly slow there. Thus, the plzen2-archive has a limit to 50000 files per user. :* for ''large files and file packages'' (tar, zip), '''choose any archive''' you like as your destination. == Transfering the files to/from the archive == {{Rámeček-green | text='''The basic rule:''' '''DON'T use front-end servers''' for anything else than moving several small files! ''Submit a regular job and/or take an interactive job'' instead to handle with the archival data. :'''''Note:''' The data transfer could be performed as a common job performing tar/rsync commands as well (see tips below). In that case, you are advised to '''delete primary data after you made sure the archival copy is OK'''.'' In both the cases, we recommend to '''submit the job to the cluster your source storage is close to''', e.g. (in the case your primary data is in Brno) <source lang=bash> qsub -I -l nodes=1:brno -l walltime=2d </source> }} {{Warning | '''The master HOME directory''' of each HSM storage (i.e. <code>/storage/plzen2-archive/home/$USER/</code>) is ''dedicated just for initialization scripts'', and thus has a '''limited quota of just 50 MB'''. To archive your data, '''use the <code>VO_metacentrum-tape_tape</code> subdirectory''' (e.g. <code>/storage/plzen2-archive/home/$USER/VO_metacentrum-tape_tape</code>), where this limitation ''doesn't apply''.}} ===Packed archives=== * '''To create a packed archive, use:''' : <code>tar czvf /storage/DESTINATION-archive/home/USER/VO_metacentrum-tape_tape/my-archive.tgz my-archive</code> :* to ''list the content'' of the archive, use :: <code>tar tzf /storage/DESTINATION-archive/home/USER/VO_metacentrum-tape_tape/my-archive.tgz</code> :* if the ''archive creation is successful'', you may delete the folder on your primary storage. * '''To recover the WHOLE data from the packed archive:''' ** ''change the current working directory'' to the place you want the data save to and use: :: <code>tar xzvf /storage/DESTINATION-archive/home/USER/VO_metacentrum-tape_tape/my-archive.tgz</code> * '''To recover a PART of the data from the packed archive:''' ** ''change the current working directory'' to the place you want the data save to and ''determine the part from the archive'' you want to recover (i.e., list the content of the archive) *: <code>tar tzf /storage/DESTINATION-archive/home/USER/VO_metacentrum-tape_tape/my-archive.tgz</code> ** further use: **: <code>tar xzvf /storage/DESTINATION-archive/home/USER/VO_metacentrum-tape_tape/my-archive.tgz "PATH1/file1" "PATH2/dir2"</code> *: OR (in the case you want to use wildcard patterns) *:: <code>tar xzvf /storage/DESTINATION-archive/home/USER/VO_metacentrum-tape_tape/my-archive.tgz --wildcards "PATH1/files*" "PATH2/dirs*"</code> ===One-to-one copies=== * '''To move the data into the archive as a one-to-one copy, use:''' : <code>rsync -avHS --no-g my-archive /storage/DESTINATION-archive/home/USER/VO_metacentrum-tape_tape/my-archive</code> :* if the command fails, just run it again (it would continue from the point it stopped). :* similarly, if you want to '''update the archive''', simply run the command again. * '''To recover the WHOLE data from the packed archive''', use: *: <code>rsync -avHS --no-g /storage/DESTINATION-archive/home/USER/VO_metacentrum-tape_tape/my_archive /storage/DESTINATION/home/USER/my_data_recovery/my_archive</code> * '''To recover a PART of the data from the packed archive:''' ** ''determine the part from the archive'' you want to recover (i.e., by listing the files in the archival storage using common <code>ls</code> command) *: <code>ls /storage/DESTINATION-archive/home/USER/VO_metacentrum-tape_tape/my-archive</code> ** and use: **: <code>rsync -avHS --no-g /storage/DESTINATION-archive/home/USER/VO_metacentrum-tape_tape/my_archive/DIR1 /storage/DESTINATION/home/USER/my_data_recovery/DIR1</code> *: OR (in the case of lower number of files to recover) *:: <code>cp -r /storage/DESTINATION-archive/home/USER/VO_metacentrum-tape_tape/my-archive/PATH1/dirs* /storage/DESTINATION/home/USER/my_data_recovery</code> == Tips == '''1. Non-interactive data transfer''' For larger data transfers one can use non-interactive way, just by modifying qsub parameters slightly: '''qsub -m abe -l nodes=1:brno''' You are prompted for command -- use a set of tar/rsync commands above, press ''enter'' and finish submition with ''ctrl+d''. PBS job is scheduled, and you will be informed via email about start/completion/error of the transfer. Standard/error outputs of the job are stored in STDIN.* files. '''2. Migration of data between any two storages''' The commands above can be also used for a migration of data between any two storages. Just change the directories to point the storages. E.g., if you want to move your data from old&slow /storage/brno2 to new&empty /storage/ostrava1. Simply use: '''rsync -avHS --no-g /storage/brno2/home/USER/my-data /storage/ostrava1/home/USER/my-data''' [[Category:Working with data]] 7cbscxuw75beqy97jcrkot9s9buxc23 0 478 2056 2051 2016-05-09T12:12:28Z Root 1 wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[Working with data]]|}} }} This is a basic tutorial how to transfer data from/to MetaCentrum based on your operating system. __NOTOC__ {{RedInfo|For working with huge amount of data, see the [[Working with data/Direct access to data storages|Direct access to data storages]] topic}} ==Data transfer on Linux== ====Moving data from your computer to MetaCentrum==== {{YellowInfo|You can replace "skirit.metacentrum.cz" with any available [[Frontend|frontend]]}} Use your terminal and '''scp''' command. Syntax of the command: <source lang="bash"> scp target_file 'META username'@skirit.metacentrum.cz: </source> ====Moving data from MetaCentrum to a local computer==== You can also use terminal and '''scp''' command. Syntax of the command: <source lang="bash"> scp 'META username'@skirit.metacentrum.cz:target_file . </source> ==Data transfer on Windows== Windows users need an ''FTPS client''. We recommend '''WinSCP''' or '''Bitvise SSH Client''' applications. You can follow the tutorials below. *[[Usage of WinSCP]] *[[Usage of Bitvise SSH Client]] ==Downloading data from a server== You can download your data from an external server to your directory via a [https://www.gnu.org/software/wget/ '''wget'''] command. For example: <source lang="bash"> 'META username'@skirit.metacentrum.cz # Login pwd # Shows current working directory /storage/brno2/home/$LOGNAME # Creates directory for data and enters it mkdir data cd data # Downloads the data wget http://www.someServer.org/myData.zip </source> [[Category:Working with data]] kjwy55z2aya66a7785bzp4pqm8ry9cx 0 551 1694 1692 2016-03-27T17:54:23Z Root 1 wikitext text/x-wiki This topic provides frequently asked questions. If you don't find an answer to your question, feel free to contact our [[:Category:User support|user support]]. * [[FAQ/General|General]] &ndash; answers to unclassifiable questions * [[FAQ/Accounts|Accounts]] &ndash; everything about accounts and login * [[FAQ/Passwords|Passwords]] &ndash; expired and forgotten passwords * [[FAQ/Accessing machines|Accessing machines]] &ndash; how to log to machines and how to use machines * [[FAQ/Job submission|Job submission]] &ndash; questions about setting job submission * [[FAQ/Application|Applications]] &ndash; problems with using applications [[Category:User support]] 6ccq2guld0ohx0kgt9kug15mbr4k7yk 0 553 1686 1685 2016-03-27T17:48:37Z Root 1 wikitext text/x-wiki ===Je pro využívání kapacit METACentra vyžadován od uživatelů nějaký finanční příspěvek?=== Využívání kapacit METACentra je bezplatné. Jedinou podmínkou je členství v akademické sféře České republiky (viz informace v sekci [http://metavo.metacentrum.cz/cs/application/index.html Přihláška]). ===Jak se v publikaci odkázat na použití zdrojů MetaCentra? Nebo také jaká je doporučená formulace poděkování MetaCentru v publikacích?=== Doporučená formulace je uvedena na stránce [[Pravidla využití MetaVO/Poděkování|Publikace a ročenka]] a pak znovu v ''Můj účet'' na stránce [http://metavo.metacentrum.cz/cs/myaccount/pubs zadávání publikací]. ===Kde je na webu MetaCentra (...) ? Nemohu to nalézt.=== Zkuste zadat hledaný výraz do okénka před tlačítkem Hledat vlevo pod navigačním menu a zmáčknou tlačítko Hledat. Google vyhledá všechny výskyty na tomto portálu. Alternativně můžete zadat hledaný výraz přímo do Google a přidat za něj <tt>site:metacentrum.cz</tt> což omezí prohledávanou oblast na stránky portálu MetaCentra. *MISS přeložit [[Category:User support]] 8g7i427pfh5jao2zlfu6pudaip2oscy 0 554 1687 2016-03-27T17:49:06Z Root 1 Založena nová stránka s textem „===How to obtain an account in METACentrum and subsequently obtain access to METACentrum machines?=== An account in METACentrum can be obtained by filling…“ wikitext text/x-wiki ===How to obtain an account in METACentrum and subsequently obtain access to METACentrum machines?=== An account in METACentrum can be obtained by filling in an [https://metavo.metacentrum.cz/osobni/prihlaska.do registration form]. A PDF file will be generated after filling the form. This PDF file has to be printed out, signed and send to a [https://metavo.metacentrum.cz/osobni/kontakty.do contact person](according to affiliation to home AFS cell). ===How to obtain access to further machines if I already have an account in METACentrum?=== If you have already a functional METACentrum account, it is sufficient to only ask for obtaining of [https://metavo.metacentrum.cz/cs/myaccount/dalsiucty other accounts].It can be required printing out and signing of a special form in the case of access to a machine with special access policy. ===What to do if it is not possible to correctly read/print generated PDF file after a successul registration?=== The form can be seen repeatedly in section [https://metavo.metacentrum.cz/osobni/seznam.do Registration state checking]. ===What does the ''Accounts activation'' mean?=== If you know in advance that you will not use some of your accounts at a specific METACentrum machines from any reasons for a long time (foreign stay, change to different architecture etc.), we offer a possibility to deactivate such accounts temporarily (e.g. to prevent their abuse). "Accounts activation" is reverse process i.e. activation of deactivated accounts. Activation/deactivation has no relation to possible expiration due to non-renewal of METACentrum account. ===Why is it necessary to renew account every calendar year?=== Because by the renewal users demonstrate their own activity performed using available METACentrum resources. Moreover, it is necessary for the user to go through the web form and confirm that: ''I confirm that my personal information in METACentrum database are actual, that I am informed about actual version of user proclamation and I do not know about any fact that would hinder me from accessing of technical resources of METACentrum.'' By this step we are trying to partially eliminate non-active users or users who do not utilize METACentrum resources in harmony with their planned mission. The METACentrum workers reserve the right to use submitted information during the preparation of actual METACentrum yearbook. ===How to renew a validity of an account that has not been used for a long time and/or for which I did not ask for a renewal and therefore I lost the access to?=== If you remember your password, ask for accounts renewal in a standard way. If you forgot your password, you have to personally visit arbitrary contact place (together with proper proving of your identity) and have to ask for [http://metavo.metacentrum.cz/en/myaccount/heslo.html setting a new password] . Subsequently you can ask for account renewal. ===How does the mechanism of an account renewal work?=== All METACentrum users are asked to renew their accounts at the end of a calendar year. This can be easily done in section [http://metavo.metacentrum.cz/en/myaccount/prodlouzeni Annual report and extension of accounts] by selection of a period for which the account should be renewed and by submitting a report containing information for what purposes the user utilized METACentrum resources during a specific calendar year. ===Do I have to submit an annual report?=== Of course, you do not. But if you do not submit the annual report you are going to get rid of possibility to use computational capacities of METACentrum. ===What will happen if I do not submit an annual report?=== If you do not submit an annual report, all your active accounts will be blocked. Such accounts can be unblocked in the future but only in the case that you submit an annual report from the last year of your active working in METACentrum. ===Do I have an account in METACentrum forever?=== The existence of your METACentrum account is limited only by your affiliation to the academic sphere. You have only to take care of the on time METACentrum account renewal (before the end of a calendar year) and of the submission of an annual report from your past calendar year. If you haveasked for an METACentrum account during your life already and you have obtained the account, it is probable that your account can be renewed even after a long period of non-activity. ===Why I can not have account at machines loslab, wood and quark?=== Cluster loslab was acquired from resources of research centre [http://loschmidt.chemi.muni.cz/peg/loadframe.html Loschmidt Laboratories] and therefore these resources are accessible only by researchers from this centre. Similar situation is for cluster wood that is dedicated to researchers of Department of Wood Science of MZLU and The Faculty of Electrical Engineering and Communication of VUT. Cluster quark is dedicated to video proceeding at MU. Due to these reasons it is not allowed to make accounts for common METACentrum users at these machines. [[Category:User support]] 8mvrvh9gafa4vf200pgfwqak6gnzj34 0 555 1688 2016-03-27T17:49:44Z Root 1 Založena nová stránka s textem „{{TODO|Chybí zde nějaký text. Jaký, to bude zřejmé ze [[Special:Whatlinkshere/Slovník - K]].}} ===What to do if I forget/lose a password to METACent…“ wikitext text/x-wiki {{TODO|Chybí zde nějaký text. Jaký, to bude zřejmé ze [[Special:Whatlinkshere/Slovník - K]].}} ===What to do if I forget/lose a password to METACentrum resources?=== Please visit the page [http://metavo.metacentrum.cz/en/myaccount/heslo.html My Account - Change password] and follow the instructions. ===Is it possible to ask for sending a new password by e-mail or other communication channel if I forget my password?=== No. Password cannot be changed based on e-mail request because METACentrum administrators have no guarantee that the e-mail comes from the same person who properly submitted and signed registration form, fulfills requirements for account granting, is authorized to use our resources and who pledged to respect our security and operational rules. Moreover the password could be eavesdropped during the transfer. You can enter a new password as a part of the ''Password change request'' that can be requested from the page [http://metavo.metacentrum.cz/en/myaccount/heslo.html My Account - Change password]. We will try to verify the request by calling over telephone to the phone number specified in the orginal user's details, or to a number acquired from some independent location like official web pages of the user's institution. If it is not possible to verify the request, you have to visit the closest METACentrum contact place and after proving of your identity to set a new password. You can consider this as unnecessary bothering but, please, take into account that in METACentrum you obtain access to the technical equipment in price of tens of millions of CZK and therefore the acces has to be secured properly. ===What is the difference between AFS password and password/passwords to individual machines?=== There is no difference. We use system Kerberos for authentication to all resources in METACentrum, i.e. AFS, METACentrum portal as well as accounts at individual machines. ===How to change a password for access to METACentrum resources?=== Please visit the page [http://metavo.metacentrum.cz/en/myaccount/heslo.html My Account - Change password] and follow the instructions. ===What does the authentication by principal @META mean exactly and how does it work?=== Principal is user identity in Kerberos system, META is realm (name space) of Kerberos system used in METACentrum. When you enter the password during log in process or using a command kauth a [[Slovník - K#Kerberos_ticket|ticket]] (lístek) with limited time validity (standard 10 hours) is created. This ticket serves for access to further resources without necessity to repeatedly enter the password. [[Category:User support]] lypvvjlh9syd0jgip0t4tn4pzys16re 0 556 1690 2016-03-27T17:51:01Z Root 1 Založena nová stránka s textem „===Why there is just queue preemptible at orca cluster?=== Cluster orca was bought using [http://ncbr.chemi.muni.cz/ NCBR] resources and it was involved in…“ wikitext text/x-wiki ===Why there is just queue preemptible at orca cluster?=== Cluster orca was bought using [http://ncbr.chemi.muni.cz/ NCBR] resources and it was involved in METACentrum in a special regime - the owners can submit any jobs through queue orca, the rest of METACentrum can submit only in queue preemptible. Queue preemptible is similar to the queue long, but its jobs can be preempted ===Why it is not possible to connect to server skirit.ics.muni.cz using SSH, WinSCP respectively?=== If you use client [[Slovník - P#Putty|PuTTy]], your configuration is not compatible with settings of our SSH servers. Please, change your client settings accordingly: the prefered SSH protocol version to 2 (instead of 1), see item ''Preferred SSH protocol version'' at ''Connection->SSH'' tab and also activate item ''Attempt "keyboard-interactive" auth (SSH-2)'' at ''Connection->SSH->Auth'' tab. ===I want to "download" my results "manually" to my /home directory or directly to my PC throught sftp (WinScp). Is it possible?=== Yes, it is. Your results can be copied into <tt>/home</tt> by command in the task script automatically. If you PC run permanently with server software (e.g. [[Slovník - S#SFTP|SFTP server]]which can be run in Windows too) and has very good connection, your results would be sent directly there, but I don't recommend it in case of large files. MetaCentrum rules says that you should have the data in a scratch volume just during task proceeding. So, it is not possible for a task to leave the data at scratch volume after it finish and after the job is done copy manually the data from scratch through [[Slovník - W#WinSCP|WinSCP]]. Realize that after end of your task, new task will be run at the same machine which maybe need space in the scratch volume. ===Is there something like cleaning /storage and /scratch directories auomatically? If so, how often? According which rules?=== Although there is an automatic script for cleaning of <tt>/scratch*</tt>, data in <tt>/scratch*</tt> must be removed by task itself. The tt>/storage</tt> filesystem is suitable for storage of permanent data. The size of the tt>/storage</tt> is limited, there are quotas on it. ===Is there any problem during copying output to /home in case the computing run at SKURUT33 machine and the computing started from /home at SKIRIT machine?=== Yes, there is problem, because <tt>/home</tt> at SKURUT is in Prague but /home at SKIRIT is in Brno. There are absolutely different directories. [[Category:User support]] 12x7era5472ivm2x2x7dtctkao76jn2 0 557 1691 2016-03-27T17:52:44Z Root 1 Založena nová stránka s textem „{{TODO|Neodpovídá anglický text české verzi.}} ===I have a problem concerning submission of a script for PBS, I am obtaining an error message: "^M: co…“ wikitext text/x-wiki {{TODO|Neodpovídá anglický text české verzi.}} ===I have a problem concerning submission of a script for PBS, I am obtaining an error message: "^M: command not found". What is the reason of this problem and how can I solve it??=== Described problem is related to the fact that your script for PBS system originated in competitive operational system (DOS/Windows) which treates line ends of a text file differently compared to Unix/Linux. Users are allowed to convert their files from DOS/Windows before submission using command <tt>dos2unix</tt> (<tt>fromdos</tt>). ===Imagine a following situation: I inform a job using stagein where the job should obtain the input (input files} and I submit the job, the job is queued and in the meantime I change the job input files in the directory. Will be the job computed with the input before the job submission or will be the job computed using the input that is stored at specific stagein address when the job starts to be executed at a CPU?=== Input files/directories are copied using stagein in the moment when the job is being executed at a specific node (there is a standard data transfer using [[Slovník - S#SCP|SCP]]) before/after job execution). I.e. if you change the input when the job is queued, the job should be computed with new input (with obvious premise that the input files have the same names). ===I want to run jobs explicitly at machine eru1.ruk.cuni.cz or at aule.ics.muni.cz. Is there a way how to specify to <tt>qsub</tt> command that I want only one concrete machine and no other?=== There is a way using command qsub -l nodes=MACHINE_NAME:ppn=NUMBER_OF_PROCESSORS It would it be more useful do not select concrete machine but arbitrary machine with 64-bit system (architecture AMD64 or EM64T), then it is better to require in <tt>qsub</tt> command property <tt>amd64</tt>. ===What does the error message "qsub: Cannot find a valid kerberos 5 ticket: TGT ticket expired" mean?=== A valid Kerberos ticket is required for successful submission of a computational job into PBS batch system. Commonly an user obtains Kerberos ticket automatically after successful login to METACentrum machine. A ticket is valid for 10 hours. If the Kerberos ticket expires and the user wants to submit a job into the system the mentioned error message will appear. This can be solved by renewal of the ticket validity by command <tt>kauth</tt> (or <tt>kinit</tt> &ndash) or by new login. Current state of your Kerberos tickets can be checked using command <tt>klist</tt>. More information can be found in [[Kerberos authentication system|Kerberos dokumentation]]. ===Is there a way how to set a script for my jobs that fell out from the queue not to be computed automatically again? (because I would prefer to submit them on my own)=== Add parameter <tt>-r n</tt> to a command <tt>qsub</tt>. ===How can I set jobs to run after a specific time period? (e.g. if I want them to prepare in advance)=== Add parameter <tt>-a</tt> to a command <tt>qsub</tt> (-a date_time Declares the time after which the job is eligible for execution). ===I compiled my program using Fortran compiler from Intel. However when it runs in a queue the error message „IOSTAT = 29“ will appear. During execution of the program in the command line there is no such error message. Is it necessary to specify paths for the input files when running in a queue?=== In the beginning of task script, which you submit to [[Slovník - P#PBS|PBS]], is good to change current directory according your needs. Otherwise your job runs in current directory of cluster, not in directory, which was actual during running <tt>qsub</tt>. (For a link to this directory from task script is possible to use a variable <tt>PBS_O_WORKDIR</tt>, if job runs on the same cluster as <tt>qsub</tt>.) It is possible to have the input/output files copied automatically to/from a computational node through parameters <tt>-W stagein=...</tt> and <tt>-W stageout=...</tt> command <tt>qsub</tt>. It is good to use AFS directory (~/shared), which is slower( particulary not so good for temporary and huge files), but visible from all Metacentrum nodes. ===According to the documentation programs should run from scratch directory. How this can be arranged if I do not have the right to write in the scratch directory?=== Your sub-directory on the scratch volume (i.e: <tt>/scratch/you_user_name/job_JOBID</tt>), identified by the <code>$SCRATCHDIR </code> environment variable, is mentioned. (You have this directory just in powerful nodes, not in leading node of cluster ( skirit.ics.muni.cz etc.)). Task script could look like this. e.g: <source lang="bash"> cd $SCRATCHDIR cp /storage/brno1/home/your_user_name/.../input* . # the computation cp output* /storage/brno1/home/your_user_name/.../ # cleaning: rm -rf $SCRATCHDIR </source> ===I have scheduled a set of jobs in a queue and then exchange the binary used for solving the jobs. Is this change taken into account for already scheduled jobs? Does PBS try to store every information needed to run a job?=== PBS hides task script and settings of variable during submitting task. The script is executed after its initiating, everything else (called programs, files for stagein etc.) is used actual. In case of binnary or other difficulty structured files change is important to focus on influencing currently running task - it is good to rename a file, runnig task could use older version without danger. cc -o program.version2 .... mv program program.version1 && mv program.version2 program or throught symbolic links, which linking to particular version cc -o program.version1 ... ln -s program.version1 program # running cc -o program.version2 ... rm program && ln -s program.version2 program ===I want to be informed via e-mail in case of job terminanting. What should I do?=== Add parameter <tt>-m ae</tt> to command <tt>qsub</tt>. All details of command qsub you can find in <tt>qsub</tt> manual page <tt>man qsub</tt>. ===How to detect where was job placed and how to log in this machine and have full possibility to check running job?=== You can see your jobs at portal and you can see there used machines and processors for [http://metavo.metacentrum.cz/pbsmon2/jobs/my each job]. Then you can log in them through [[Slovník - S#SSH|SSH]] ([[Slovník - P#Putty|PuTTy]]) and check your jobs as you want. ===I need to solve automatic delete of working directory in case of computing interuption (because I have the proper command to delete working directory at the end of the script).=== If it is computing interuption because of limit exceeding, you can handle it by setting the task to respond to SIGTERM signal and remove the files in this case. The catching of signals is defined in shell script by command <tt>trap</tt>, e.g. trap "clean_scratch" SIGTERM It will work also in case of interuption of computing by command <tt>qdel</tt>. Cases when the crash of job is made by serious system error (e.g. blackout) can not be handled automatically. ===If I run 2 GPU jobs on the same node, 1 GPU per each. How I recognize which GPU belongs to the job?=== Type this command: <tt>/usr/sbin/list_cache arien gpu_allocation | grep $PBS_JOBID</tt> [[Kategorie:User support]] 4jpwsn8k1fahl8devl92hf8ok4mstdf 0 558 1693 2016-03-27T17:54:04Z Root 1 Založena nová stránka s textem „===I can not run queen properly at testing example (see /home/pavelm/queen/test)=== In case I run queen locally in directory /home/pavelm/queen/test (by…“ wikitext text/x-wiki ===I can not run queen properly at testing example (see /home/pavelm/queen/test)=== In case I run queen locally in directory /home/pavelm/queen/test (by command "queen --Iuni example noe") the job runs properly, in case I want to queue the job, it fails with error message: "ERROR - Dictionary /home/pavelm/queen/test/queen.conf could not be read. Script has been stopped." I tryied to run this script by psubmit and also by qsub in normal queue ("qsub -q normal run" nebo "psubmit normal run"). Script contains both "metamodule add queen" and setting of path to configuration file into environment variable QUEEN_CONF ("QUEEN_CONF=/home/pavelm/queen/test/queen.conf;export QUEEN_CONF"). Script to execute is following: metamodule add queen Q_PROJECT=/home/pavelm/queen/test/ export Q_PROJECT QUEEN_CONF=/home/pavelm/queen/test/queen.conf export QUEEN_CONF #export Q_PROJECT=/home/pavelm/queen/test/ #export QUEEN_CONF=/home/pavelm/queen/test/queen.conf queen --Iuni example noe Unfortunatelly I can not learn why my configuration file can not be read Path which is set into QUEEN_CONF must be available from machine, where PBS ran your job (in case of parallel jobs from all nodes), so path /home/pavelm/... is local for clusters (leads on different clusters to different disks) Either put queen.conf into your directory in AFS(/afs/ics.muni.cz/home/pavelm, or /home/pavelm/shared) this directory is seen from all MetaCentrum machines, or limit running of job just to machines, which have access to your directory. E.g: by option PBS -l brno (-l ...:brno) if you have it in /home on skirit or perian. ...For program efective running you have to put subdirectory "log" in assigment (item Q_LOG in queen.conf) into the fastest storage -- e.g. on a scratch volume (in a directory dedicated for a job -- identified by the $SCRATCHDIR environment variable) or in a /storage volume. ===I linked through Q_LOG on $SCRATCHDIR ... Can I apply above described use of $SCRATCHDIR to parallel job in program xplor-nih.=== According documentation (/afs/ics.muni.cz/software/xplor-nih-2.20/parallel.txt) you have to have a directory shared by all used nodes using program xplor in parallel mode. The best way should be to use the shared scratch volume (available just at the mandos cluster -- the dedicated space is identified by the $SCRATCHDIR environment variable), or to use a NFSv4 volume within the /storage directory. By the way, QUEEN must be able to read assingment from all nodes (e.g. it could be copied for just for reading), but subdirectory log could and it is sufficient to be on each node local, because some parts of a job run in it as separely runned xplor instance which communicate with queen local processes through files. Result of whole job is written just by main process. =Gaussian= ===Memory alocation in program Gaussian version G03.E01 does not work. What should I do?=== In case you get warning: "buffer allocation failed in ntrext1." just after running the program, use a directory on a scratch volume (the directory is identified by the $SCRATCHDIR environment variable) as a working directory instead of the /home (there is an error in the new version, that occurs according type of file system) ===I think there is a problem with the node orca14-2. Gaussian computings have fallen down immediately after running with error message (every time in different part of the log file) ´´Erroneous write. Write 8192 instead of 12288.´´=== Reason of this failure is that your disk is full of undeleted Gaussian helping files. These files are remains from your previous computings which weren't successful. You can avoid this situation by deleting all helping files after the end of computing. The easiest way is to create a own directory for every single computing and to remove this directory in the end of computing. GAUSS_SCRDIR=$SCRATCHDIR GAUSS_ARCHDIR=$GAUSS_SCRDIR mkdir $GAUSS_SCRDIR ... g03 ... rm -rf $GAUSS_SCRDIR [[Category:User support]] 6t2plxl9dkrbd4k15j1hsqmqph76au8 14 533 1773 1524 2016-03-30T10:18:52Z Root 1 wikitext text/x-wiki This category contains the collection of policies used for various MetaCentrum activities. [[Category:User support]] ff9tizg87ospzby6nulejz6kkryyehb 0 588 5255 2083 2016-05-13T17:59:29Z Root 1 /* Inserting a source code */ wikitext text/x-wiki This topic describes how to use this documentation and rules for writing a new topic. {{YellowInfo|You have to be logged in to see '''Edit''' button in the upper bar, which shows source code of a topic}} ==How to use documentation== This documentation is prepared on platform '''MediaWiki''', which brings various elements for users. Besides the main elements like upper bar, left navigation menu, search box etc., there are also available templates, links, special pages and many more. Following text shows how to use them. ===Linking=== You can link with following commands: {| class="wikitable" ! style="background-color:LightSeaGreen;"|Command !! style="background-color:LightSeaGreen;"|Description !! style="background-color:LightSeaGreen;"|Example |- | <source lang="php> [[TargetTopic]]</source> || Link to target topic || [[Usage rules]] |- | <source lang="php> [[TargetTopic|Alternative text]]</source> || Link to target topic has an alternative text || [[Usage rules|UR]] |- | <source lang="php> [[:Category:TargetCategory]]</source> || Link to target category || [[:Category:Grid services]] |- | <source lang="php> [[:Category:TargetCategory|Alternative text]]</source> || Link to target category has an alternative text || [[:Category:Grid services|Grid services]] |- | <source lang="php> [[TargetTopic#TargetHeading]]</source> || Link to a heading in target topic || [[How to write a new topic#Linking]] |- | <source lang="php> [[TargetTopic#TargetHeading|Alt. text]]</source> || Link to a heading in target topic has an alt. text|| [[How to write a new topic#Linking|Linking]] |- | <source lang="php> [http:targetAddress Alternative text]</source> || Link to target webpage || [https://www.mediawiki.org/wiki/MediaWiki MediaWiki] |- |} ===Text formatting=== ''Font formatting'' is possible with following commands: * '''Bold text''' &ndash; <nowiki> '''Bold text'''</nowiki> * ''Italics'' &ndash; <nowiki> ''Italics''</nowiki> * <span style="color:green">Coloured text</span> &ndash; <nowiki> <span style="color:green">Coloured text</span> </nowiki>. You can use other colours as well (red, yellow ...). ''Text formatting'' is possible via headings, lists and tables. There are more possibilities of lists and tables than shown below, see [https://www.mediawiki.org/wiki/Help:Lists Lists] and [https://www.mediawiki.org/wiki/Help:Tables Tables] for more info. ====Inserting a source code==== If you would like to insert a source code, use the '''<nowiki> <source lang="targetLanguage"> source code </source> </nowiki>''' tag. ====Headings==== {{YellowInfo|It is usually better to skip the first level headings in a topic}} Headings are created with "'''='''" character and you can create up to seven levels of headings. The deeper a heading is, the smaller it is. For example: <source lang="php"> == Heading 1 == == Subheading 2 == ==== Subheading 3 ==== </source> ''Table of contests'' is automatically generated when a topic has at least 3 headings. You can add it manually to topic with 1-2 headings with the <nowiki>__TOC__</nowiki> command. On the other hand, you can manually remove table of contests with the <nowiki>__NOTOC__</nowiki> command. ====Lists==== Bullet lists are created with "'''*'''" character and numbered lists with "'''#'''" character. For example: <source lang="php"> Bullet list ----------- * Item1 * Item2 Numbered list ----------- # Item1 # Item2 </source> ====Tables==== You can add a simple table with the following code: <source lang="php"> {| class="wikitable" !|Column1|Column2|...|ColumnN |- (new line character) |Value1||Value2||...||ValueN |- |Value1||Value2||...||ValueM |} </source> Tables can be additionally edited with CSS. ===Graphical and navigation elements=== ====Images==== You can insert an image via the <nowiki>[[Image:TargetImage]]</nowiki> command. First, you have to upload the image via'''Upload file''' button in the upper bar. ====Highlighting labels==== There are three labels of this type - information label, warning label and general label. You can insert them into a topic with following commands: <source lang="php"> {{YellowInfo|TargetText}} -- Inserts information label (yellow) {{RedInfo|TargetText}} -- Inserts warning label (red) {{GreenBox|TargetText}} -- Insert general label (green) </source> ====Navigation boxes==== You can add navigation box into a topic with following code: <source lang="php"> {{Box|BoxHeading {{BoxPoložka|TargetLink1|TargetLink2}} }} </source> Every <nowiki>{{BoxPoložka|TargetLink1|TargetLink2}}</nowiki> line adds new line into the box. You don't have to specify both links, usually the first one is enough &ndash; the second one can be empty. See the [[How to compute]] topic for example of the box. ====Subpages==== You can create subpage (=subtopic) of a topic with correct title of the subpage. When you are creating target subpage, name it in format '''parentPage/subPage'''. This is the only way how to create a subpage. See [[How to compute/Quick start]] for an example of subpage. [[Category:User support]] cejdbh4ccuzsxp6hn6iihgx8yqvxe2k 0 539 5249 5245 2016-05-13T15:54:13Z Root 1 wikitext text/x-wiki ([[Registrace do Hostel IdP|Česká verze]]) [https://hostel.eduid.cz/en/overeni_identity.html Hostel Identity Provider (IdP)] is a service for users from institutes, which are '''not members of czech identity federation''' [http://www.eduid.cz/en/index eduID.cz] yet. Verification through the federation is easier than through Hostel IdP. In case you work for an institution that is involved in eduID.cz, you don't have to use Hostel Idp. If you do not have another option, use this service (Hostel IdP) provided by CESNET. The registration to Hostel IdP is done during [http://metavo.metacentrum.cz/cs/myaccount/prodlouzeni account extension] or [http://metavo.metacentrum.cz/cs/application/index.html registration], if you are from an institution, which is not part of eduID.cz. [[Soubor:Hostel-logo.png]] Registration to Hostel IdP is 4-step process (all steps are essential for working MetaCentrum account): * [http://www.eduid.cz/en/members Check] if your organization is member of eduID.cz. * '''Registration / login to Hostelu IdP''' ** [https://adm.hostel.eduid.cz/meta2hostel/ existing MetaCentrum users have own interface for account activation] in Hostel IdP, '''they login with their MetaCentrum username and password'''. ** others use their email address for registration * '''One-time proof of identity''' (ID or passport + study or employment relationship certificate) with registration official - it it necessary for access to premium services (computing in MetaCentrum, data storages,...). * Regularly (usually 1x per year) '''confirmation of study or an employment relationship''' submission. You can find '''more information, tutorials and contacts''' on [https://hostel.eduid.cz/en/overeni_identity.html Hostel IdP service webpage]. [[Category:User support]] [[Category:Policies]] hcxtzy902biyjtl56zxnubnoaczpylp 0 2217 5254 5253 2016-05-13T16:29:13Z Root 1 /* EduID Hostel IdP verification */ wikitext text/x-wiki You can personally contact following persons in Prague: * [http://www.cesnet.cz/cgi-bin/hledkont?jmeno=hanousek Petr Hanousek] &ndash; general MetaCentrum user support. You can find him in CESNET headquarters (Zikova 4, Praha Dejvice, over ČVUT rector's office), office 90a. * [http://www.cesnet.cz/cgi-bin/hledkont?jmeno=chudoba Jiří Chudoba] &ndash; grid applications support, primary for VO ALICE, ATLAS, AUGER, BELLE, CTA (vo.cta.in2p3.fr) and contact for cluster Luna. You can find him in psychical institute of AVČR (Na Slovance 1999/2, 182 21 Praha 8). <span title="Zikova 13 v Dejvicích">[[Soubor:Metacentrum praha.png|left|400px|link=http://mapy.cz/s/8WAV]]</span> <span title="FZU v Libni">[[Soubor:Fzu praha mapa.png|400px|link=http://mapy.cz/s/3n3Q]]</span> ==EduID Hostel IdP verification== Verification for Hostel IdP lze is possible in two places in Prague: *[https://hostel.eduid.cz/cs/overeni_identity_cesnet.html Registration office] [http://www.cesnet.cz CESNET]. You can find him on the third floor of ČVUT rector's office across Sinkule's college. *[https://hostel.eduid.cz/cs/overeni_identity_fzu.html Registration office] [http://www.fzu.cz/ Psychical institute of AVČR]. [[Category:User support]] 3xpax511unk677glysublu1qbpckvsv 0 428 1556 1013 2016-03-09T17:10:30Z Root 1 wikitext text/x-wiki In this manual we describe major extension of the open source TORQUE Resource Manager system. We have replaced the naive scheduler provided in the TORQUE distribution with the complex scheduling system that allows to plan job execution ahead and predict the behavior of the system. It is based on the application of job schedule, which represents jobs' execution plan. Such a functionality is very useful as the plan can be used by the users to see when and where their jobs will be executed. Moreover, created plans can be easily evaluated in order to identify possible inefficiencies. Then, repair actions can be taken immediately and the inefficiencies can be fixed, producing better schedules with respect to considered criteria. This manual expects that you are familiar with the original TORQUE Admin Manual available on: http://www.clusterresources.com/torquedocs21/. First we describe the princip of the plan-based approach. In the next this manual describes the difference to the the TORQUE Resource Manager used in Czech Nation Grid - MetaCentrum. Another part deals with an installation of this software and how to set up the software. We also describe the way how to submit a job in to the plan-based scheduler. And finally for those who want to know more about a plan-based scheduling this manual describes how the scheduler works. = About a Plan-Based Scheduling = We have developed working implementation of job schedule that can be used to plan job execution onto one or more computing sites such as computer cluster. The schedule is created subject to dynamically arriving jobs. Then it is used to schedule jobs on the available computing resources. The schedule is continuously maintained in order to remain consistent with the changing situation in the system. Thus all important events such as job arrivals and (early) job completions or machine failures and restarts are reflected in order to keep the schedule up-to-date with respect to the changing situation. Moreover, the users can now query the scheduler to get information from the job schedule. It means that they can ask the scheduler when and where their jobs will be executed and the scheduler provides them such a prediction according to the current job schedule. Also, several evaluation criteria has been implemented allowing the use of schedule optimization techniques. The usual schedulers are so called queueing systems. It means that these systems typically follow the queue-based scheduling approach, using one or more incoming queues where jobs are stored until they are scheduled for execution. A scheme of a basic (local) queueing system is shown in the following figure (left). Plan-based systems represent significantly different approach with respect to the queue-based solutions. Plan-based systems use job schedule as a more complex data structure that maps jobs onto available machines in time. This schedule represents de facto a plan of future job execution. In the same fashion as, e.g., backfilling algorithms require information about expected job runtime, also the plan-based solutions need such information to construct the schedule. A scheme of the general plan-based system is shown in the figure (right). There are no incoming queues that would store the jobs (actually there is one default queue, but the scheduler ignore it). Instead of that a two dimensional data structure is being built that assigns to each job its own space and time slot on given machine(s). The x-axis represents system time and the y-axis represents particular machine on a particular cluster. There are other aspects that differentiate the design of plan-based systems from the queue-based systems. The major difference is that nontrivial scheduling decision must be taken every time some new job arrives. Such immediate decision is necessary to find suitable place for the job in the schedule. Queue-based methods usually perform such decision when the job is selected for execution, i.e., at the "last possible moment" (when machine(s) become available). An exception represent those queue-based methods that use reservations. When machine(s) become available, scheduling decisions are trivial for plan-based methods. At that point in time, scheduler simply sends on such machine(s) those jobs that are stored in the schedule on corresponding coordinates. [[Soubor:Planbased-queuebased_systems.png|center]] = Optimization = One of the advantage of the plan-based apprach is that it is possible to evaluate the plan and to make decision which plan is better. In order to use this advantage we regulary lunch an optimization algorithm. This algorithm optimize the plan with respect to several criteria together. The criteria are folowing: * Response time - The avg. response time represents the average time a job spends in the system, i.e., the time from its submission to its termination. * Wait time - The avg. wait time is the mean time that the jobs spend waiting before their execution starts. * Bounded slowdown - The avg. bounded slowdown is the mean value of all jobs' bounded slowdowns. Slowdown is the ratio of the actual response time of the job to the response time if executed without any waiting. If a job has a very small runtime it often means that the job ended prematurely due to some error. As a result, its slowdown can be huge, which may seriously skew the final mean value. Therefore, bounded slowdown is applied, where the minimal job runtime is guaranteed to be greater than some predefined threshold, e.g., 10 seconds. * User-to-user fairness - This criterion is optimized by minimizing the mean and standard deviation of Normalized User Wait Time (NUWT). For a given user, NUWT is the total user wait time divided by the amount of previously consumed system resources by that user. Multiple consumed resources are reflected when calculating NUWT. All the criteria are to be minimized. Local search is used for optimization. Local search is a heuristic algorithm which improves the solution step by step. An initial solution is available for a run of the local search algorithm. The local search algorithm improve the solution in an iterative way. It finds an improvement and then the algoritm decides if this improvement is accepted or declined in each iteration. In our solution there is the improvement always accepted. The following figure describes the applied metaheuristic: [[Soubor:Optimization.png|center]] = The difference to the TORQUE scheduler in MetaCentrum = The whole scheduler is different and some features are not supported yet. == Not supported features == * The plan-based scheduler is not able to handle with admin jobs. * There is no P2P support opposed to the scheduler in MetaCentrum. * The plan-based scheduler doesn't check whether a node has a "no_multinode_jobs" property set to "true". This property can forbid multinode jobs on some node. * Cloud jobs are also not supported. * Suspend and resume of jobs (preemption of jobs) is not supported yet. * The plan-based scheduler doesn't check whether the user's account is present on the target node. <!-- * There is no way how to set the node into maintenance mode. The node is either UP or DOWN. --> == Not supported qsub options == * If you use -a option in the qsub command then the job will not be added to the schedule until the job become eligible for execution. * Job priority is not supported. This means that it is useless to use the -p option in the qsub command. * It is possible to ask for licenses but the scheduler can not guarantee that licenses will be available at planned start time (There is no reservation service for allocating licenses). The job will wait for the licenses. The job will run when all licenses will be available (after planned start time). * It is not possible to use complex submit arguments. The scheduler is not capable to handle following submit arguments: ... -l nodes=1:ppn=1+2:ppn=4 ... -l nodes=1:ppn=4:manwe3.ics.muni.cz * It is not possible to ask for a specific node or exclude a specific node. * Planning scratch is not supported the scheduler can not guarantee that scratch will be available at planned start time. * Some differences occur when you use qsub option: "-W additional_attributes". It is not possible to use some of options for depended jobs: synccount:count syncwith:jobid on:count * On the other side, following options works but the job is not added into the schedule until the dependency is meet. after:jobid[:jobid...] afterok:jobid[:jobid...] afternotok:jobid[:jobid...] afterany:jobid[:jobid...] before:jobid[:jobid...] beforeok:jobid[:jobid...] beforenotok:jobid[:jobid...] beforeany:jobid[:jobid...] = Compilation, installation and setup = On Debian and Ubuntu you need to install following packages first: apt-get install krb525 libtool Once you have unpacked a source code you can can customize the scheduler. Edit: /src/torque/src/scheduler.meta/sched_config.h * The option OPTIM_CYCLES_LIMIT is an expression which determines how many attemps to find an improvment will run in one optimization cycle. num_queued is number of jobs in the schedule which are waiting for run. #define OPTIM_CYCLES_LIMIT (num_queued * 40) * The option OPTIM_TIMEOUT is a timeout in seconds which says how often the optimization will run. #define OPTIM_TIMEOUT 30 * The option OPTIM_DURATION_LIMIT is a limit in seconds and after this limit the optimization is stopped in spite of the limit OPTIM_CYCLES_LIMIT. #define OPTIM_DURATION_LIMIT 10 * The option OPTIM_MINIMAL_QUEUED says how many jobs have to be waiting for run in order to do optimization. #define OPTIM_MINIMAL_QUEUED 3 Once you are finnished you can compile the TORQUE with following commands: cd torque autoreconf -ivf mkdir _presunuto cd magrathea/pbs_cache make cd ../.. mv magrathea/pbs_cache/api.h _presunuto/api.h mv magrathea/pbs_cache/libpbscache.a _presunuto/libpbscache.a If you are going to use TORQUE without cerberos you shoud use following CFLAGS: CFLAGS="-D_GNU_SOURCE -ggdb3 -std=gnu99 -D_XOPEN_SOURCE=700" CXXFLAGS="-D_GNU_SOURCE -ggdb3" ./configure --with-debug --with-rcp=scp --with-pbs-cache=`pwd`/_presunuto --enable-maxdefault --enable-meta-fork --prefix=/usr --disable-syslog --disable-gcc-warnings or if you want to use cerberos: CFLAGS="-D_GNU_SOURCE -ggdb3 -std=gnu99 -D_XOPEN_SOURCE=700" CXXFLAGS="-D_GNU_SOURCE -ggdb3" ./configure --with-debug --with-rcp=scp --with-pbs-cache=`pwd`/_presunuto --enable-maxdefault --enable-meta-fork --prefix=/usr --enable-syslog --disable-gcc-warnings --with-gssapi_bin=/usr/bin/ --with-gssapi_lib=/usr/lib/heimdal --with-gssapi_inc=/usr/include/heimdal and then run the compilation: make after that you can install the TORQUE: make install or prepare a packages and install the packages: make packages The next step is to set up the torque server and moms. Please, use only one queue. This procedure is described in the standard TORQUE Admin Manual but there is one more thing to do. You need to set up the scheduler's server. Please edit this file and write down the pbs server's fqdn on the appropriate line: /var/spool/torque/sched_priv/sched_config = How to submit a job = Once you have finnished the setup and the pbs_server, pbs_scheduler and pbs_moms are running you can try to run a job. * '''PLEASE ATTENTION!''' Never use a queue when you submit a job. There is only one queue which is default. The scheduler ignores the queues (an exception are priority queues dedicated to ser groups according to explicit agreements). * '''PLEASE ATTENTION!''' Always use a walltime and mem when you submit a job. This is mandatory for the plan-based scheduler. It is possible only simple nodes specification like following: qsub -l nodes=1:ppn=1:mem=409600kb,walltime=1000 qsub -l nodes=2:ppn=1:mem=40gb,walltime=10d qsub -l nodes=1:ppn=2:mem=800mb,walltime=36h It is also possible to use interactive job: qsub -l nodes=1:ppn=2:mem=4gb,walltime=2h -I The nodes are organized in to the clusters according to their properties. All nodes have the same properties within cluster. You can demand the properties. For example (linux and x86): qsub -l nodes=1:ppn=1:linux:x86:mem=409600kb,walltime=1000 <!--If you want to use scratch then it is possible to use only local scratch (by using this option the variable SCRATCHDIR become available in the job): ... -l scratch=350mb:local--> In this version of TORQUE the qstat command has been modified. Once the scheduler get the information when and where a job will run you can list this information using qstat command. There are two new lines: "planned_start" which says when the job will run and "planned_nodes" which says where the job will run. The example of the command is on the following figure: [[Soubor:Qstat.png|center]] We also use a graphical interface for displaying the schedule overview (http://metavo.metacentrum.cz/schedule-overview/) but it is more complex tool and its description is beyond this manual. The examle is on figure: [[Soubor:Sc-schedule-overview.png|center]] = How the scheduler works = In order to describe how the scheduler works we need to know how the data structure for the plan looks like. In reality, the Grid typically consists of one or more sites such as computer clusters. For each cluster we would create a separate instance of schedule. For simplicity let us assume that the Grid consists of a single cluster so there is only one schedule instance. Intuitively, the schedule defines when and where jobs will be executed, introducing a two dimensional rectangular representation such that for each job the set of assigned CPUs and the time interval is specified. Here the x-axis represents the time and the y-axis represents the CPUs of the system. An example of such a schedule is shown in figure (left). [[Soubor:Schedule_plan-based-scheduler.png|center]] In the figure (middle and right) is depicted the reprezentation of the intuitive schedule. The schedule is represented as a linear list of jobs (job_list) and a list of so called gaps (gap_list) that represents unused computational time. Gaps and jobs in these lists are ordered according to their expected start times. Each job in this list stores its start time, completition time, used CPUs... Simply, complete information are stored in a single cell of the list. All these parameters are computed when the job is added into the schedule. Of course, as the schedule can change dynamically in time, the parameters can be lately updated according to the new situation. Gaps represent unused periods of CPU time and the amount of free RAM across nodes within a cluster. Gaps appear every time the number of available CPUs in the existing schedule is greater than the number of CPUs requested by the job(s) in the given time period. Similarly, gap can appear when the amount of requested RAM is higher than it is currently available in the given time period. Similar to jobs, gaps hold information about their start time, duration, usage and free RAM. Here, the usage expresses the number of available (idle) CPUs in this gap. It is stored an array of such available CPUs with a pointer to appropriate node and the node's free RAM. Moreover, each gap has a pointer to the nearest following job. If there is no following job (gap at the end of the plan) this pointer is set to null. Two successive gaps in the gap_list have either different usage, contain different nodes, contain different amount of free RAM for the given node or the completion time of the earlier gap is smaller than the start time of the later gap. Otherwise, such gaps are merged into a single, longer gap. == New job arrival == A backfill-like procedure is used to build the schedule, i.e., to add newly arriving job into the currently known schedule. It finds the earliest suitable gap for the new job. In this case, the applied data representation represents major benefit as all gaps in the current schedule are stored in a separate list (gap_list) which speeds up the whole procedure. Two or more adjacent gaps can be used for the new job if a single gap is not large enough. In such case following constraints must be met. All gaps must at least have the same usage and available RAM memory as the job requires. Also the completion time of the preceding gap must be equal to the start time of the next gap. Finally, the duration of the job must be less than or equal to the total duration of the considered gaps. This "gap-filling" approach is very useful as it significantly increases system utilization while respecting the start times of previously added jobs. == Update schedule == Often the schedule must be updated as events such as (early) job completions, machine failures or arrival of job appear. In such situation the shceduler launches a time efficient schedule update procedure that updates the internal jobs' parameters in the job_list while recomputing the gap_list as well. Commonly, job finishes earlier than expected as the schedule is built using processing time estimates which are typically overestimated. In such case, the schedule is immediately compressed. The compression shifts jobs into earlier time slots that could have appeared as a result of an early job completion. During the compression, the relative ordering of job start times is kept. = Publications releated to this plan-based scheduler = * CHLUMSKÝ, Václav, Dalibor KLUSÁČEK a Miroslav RUDA. Planning, Predictability and Optimization within the TORQUE Scheduler. Planning, Predictability and Optimization within the TORQUE Scheduler. In Antonín Kučera, Thomas Henzinger, Jaroslav Nešetřil, Tomáš Vojnar, David Antoš. ''MEMICS 2012''. první. Brno: Novpress s.r.o., 2012. s. 96-97, 2 s. ISBN 978-80-87342-15-2. * KLUSÁČEK, Dalibor, Václav CHLUMSKÝ a Miroslav RUDA. THE EXTENSION OF TORQUE SCHEDULER ALLOWING THE USE OF PLANNING AND OPTIMIZATION IN GRIDS. THE EXTENSION OF TORQUE SCHEDULER ALLOWING THE USE OF PLANNING AND OPTIMIZATION IN GRIDS. ''Computer Science Journal'', Krakow, Poland: AGH University of Science and Technology, 2012, roč. 13, č. 2, s. 5-19. ISSN 1508-2806. doi:10.7494/csci.2012.13.2.5. * CHLUMSKÝ, Václav, Dalibor KLUSÁČEK a Miroslav RUDA. The Extension of TORQUE Scheduler Allowing the Use of Planning and Optimization Algorithms in Grids. The Extension of TORQUE Scheduler Allowing the Use of Planning and Optimization Algorithms in Grids. In ''Cracow Grid Workshop''. 2011. * KLUSÁČEK, Dalibor, Václav CHLUMSKÝ a Hana RUDOVÁ. Optimizing User Oriented Job Scheduling within TORQUE. Poster on SuperComputing 2013 at Colorado. = Contacts = * Václav Chlumský, email: vchlumsky@cesnet.cz * Dalibor Klusáček, email: xklusac@fi.muni.cz [[Category:Scheduling system]] 5lbwvk39c6vb1v71fa903b767wjheex 0 545 1638 2016-03-17T18:09:26Z Root 1 Založena nová stránka s textem „ To write GPU accelerated programs, one will need to be familiar with high-level programming languages. Most GPU programming is based on the C language and…“ wikitext text/x-wiki To write GPU accelerated programs, one will need to be familiar with high-level programming languages. Most GPU programming is based on the C language and its extensions. In the wider context, having a background in parallel computing techniques (threading, message passing, vectorization) will help one understand and apply GPU acceleration. = Submiting GPGPU jobs = * GPU clusters available: ** ''doom1.metacentrum.cz - doom30.metacentrum.cz''[http://metavo.metacentrum.cz/pbsmon2/resource/doom.metacentrum.cz] -- 2x nVidia Tesla K20 5GB (aka Kepler) per each node ** ''konos1.fav.zcu.cz - konos10.fav.zcu.cz''[http://metavo.metacentrum.cz/pbsmon2/resource/konos.fav.zcu.cz] -- 2x GPU NVIDIA GeForce GTX 465 per each node ** ''gram1.zcu.cz - gram10.zcu.cz''[http://metavo.metacentrum.cz/pbsmon2/resource/gram.zcu.cz] -- 4x nVidia Tesla M2090 6GB per each node * GPU queues: ''gpu'' (24 hours max) a ''gpu_long'' (both with open access for all MetaCentrum members) * GPU jobs on the ''konos'' cluster can be also run via the priority queue ''iti'' (queue for users from ITI - Institute of Theoretical Informatics, Univ. of West Bohemia) ==Requesting GPUs== The key scheduling constraint is to prevent jobs from sharing GPUs. To ensure this always use the gpu=''X'' flag in qsub and '''request one of the gpu queues''' (gpu, gpu_long, iti). -l nodes=1:gpu=''X'' -q gpu where ''X'' means number of GPU cards required. By default resources_default.gpu=1 If a job requires more GPU cards than it asks (or is available), prolog does not run it. == Example == qsub -q gpu -l nodes=1:gpu=1 -I Interactive job requests 1 machine and 1 GPU card in the max 24 hours queue. qsub: waiting for job 1477670.arien.ics.muni.cz to start qsub: job 1477670.arien.ics.muni.cz ready konos10.fav.zcu.cz$ env|grep GP TORQUE_RESC_GPU=1 konos10.fav.zcu.cz$ env|grep TOR TORQUE_RESC_TOTAL_VMEM=1099511627776 PBS_VERSION=TORQUE-2.4.75 TORQUE_RESC_TOTAL_WALLTIME=86400 TORQUE_RESC_VMEM=1099511627776 TORQUE_RESC_GPU=1 TORQUE_RESC_TOTAL_NODECT=1 TORQUE_RESC_TOTAL_PROCS=1 TORQUE_RESC_PROC=1 TORQUE_RESC_TOTAL_MEM=419430400 TORQUE_RESC_MEM=419430400 konos10.fav.zcu.cz$ ls -l /dev/nvidia* crw------- 1 ruda root 195, 0 Sep 20 04:00 /dev/nvidia0 crw------- 1 ruda root 195, 1 Sep 20 04:00 /dev/nvidia1 crw-rw-rw- 1 root root 195, 255 Sep 20 04:00 /dev/nvidiactl konos10.fav.zcu.cz$ exit =FAQ= Q: How to recognize which GPU card is dedicated to the job A: Information can be retrieved by following command /usr/sbin/list_cache arien gpu_allocation | grep $PBS_JOBID Q: How can I recognize which GPUs are reserved for me by planning system? A: IDs of GPU cards are stored in CUDA_VISIBLE_DEVICES variable. These IDs are mapped to CUDA tools virtual IDs. Though if CUDA_VISIBLE_DEVICES contains value 2,3 then CUDA tools will report IDs 0,1. [[Category:Infrastructure]] [[Category:Scheduling system]] kbpdxchaytoim4bfxhlv9tm29kx71re 0 562 1993 1989 2016-05-04T11:28:25Z Root 1 /* What resources are supervised */ wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[How to compute/Requesting resources|Requesting resources]]|}} {{BoxPoložka|[[About scheduling system]]|}} }} This page describes the possible reasons for an unusual job ending. The [[About scheduling system|scheduling system]] '''kills the job''' in the situations when the job exceeds memory, processors or time length limits. If you can't find answer to your question it this topic, try to read [[How to compute/Requesting resources|Requesting resources]] and [[About scheduling system]]. =What resources are supervised= * walltime &ndash; how long is the job running. It cannot run longer than the defined [[How_to_compute/Requesting_resources#Queues|queue time]] or than is specified in parameter ''walltime''. * used CPUs &ndash; the job is killed when it computed on more processors than were granted by the scheduling system. * used memory &ndash; the memory occupied by the running job. ** '''Note:''' Qsub recognizes two parameters regarding memory - ''mem'' and ''vmem''. For running you jobs please use only ''mem'' parameter, ''vmem'' (= mem + swap) is only for special purposes which, the most probably, you will not meet. The default of allocated resources when no qsub parameter is specified, is queue normal 1 node 1 processor 400MB of memory =Recommendations= ===You forgot to set some limit=== If you don't specify the queue and limit of memory, processors and nodes, the default stat of the job is queue <tt>normal</tt>, 400MB of memory, 1 node and 1 processor. More about qsub parameters can be found at [[How to compute/Requesting resources|dedicated page]]. You should always set the number of nodes, processors, memory limit and scratch size. ===Tuning up the resources needs on the test job=== For the first try to run one job of the specific type for the test and check it's needs. Other job of the same type can be run afterwards with tuned parameters. It is worth to know hom much memory and disc space the job needs before you run several jobs and system kills them all. You can see details of your jobs in the [http://metavo.metacentrum.cz/pbsmon2/person PBSmon] including it's specified and consumed resources. Unfortunately PBSmon shows the jobs quite complex so it does not show the consumed resources on each used node but in the whole. You can find more in section [[#Requests don't count with number of machines|Requests don't count with number of machines]]. At the beginning is good to demand more resources and then lower the needs. It can take longer to gather required resources for the test job, but the other jobs will be planned faster. You can see the computing needs of your job by planning the [[How to compute/Interactive jobs]] at the machine with sufficient resources and observe how many threads runs on what processors (<tt>htop</tt>). This can be done quite quickly, the job don't have to end, you can kill it after several seconds. The most frequent errors are in wrong run of parallel jobs and in wrong set up of certain programs (Java, Matlab). You can find more by reading the documentation for [[:Category:Applications|specific applications]]. The memory needs of the job can be found in [http://metavo.metacentrum.cz/pbsmon2/person PBSmon] after the end of the job because it can vary during the computation. ===Requests don't count with number of machines=== You have specified more nodes in <tt>qsub</tt> (ie. <tt>-l nodes=2:ppn=3 -l mem=4gb</tt>) but you forgot that memory is allocated for one machine. In other words for your job is reserved 8GB of memory and 6 processors, but on the 2 machines so the job can not exceed 4GB of memory and 3 processors at one machine. This principle is influenced by license needs of various SW, such as [[Gaussian-GaussView application#Availability|Gaussian]]. ===The most frequent wrongly used programs=== Some programs are more likely to exceed the requested resources regarding the memory limits. It is mainly [[Running an application on JAVA platform|Java]] and [[Matlab application|Matlab]]. The examples of correct usage of these programs are here in wiki. [[Category:Grid services]] [[Category:Scheduling system]] 6i6ocax6n4yrc1rgdvt0ubah2ksga1b 0 563 5291 5290 2016-05-16T07:11:52Z 127.0.0.1 /* Properties of computational nodes */ wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[:Category:Scheduling system|Scheduling system]]|}} {{BoxPoložka|[[Resources request examples]]|}} }} Applying for resources is possible via '''qsub''' command and its '''options''' separated by '''-l''' character. General syntax of the command is: qsub -l option1 -l option2 -l option3 ... ==Basic options== * ''maximal duration of a job'' - is set by <tt>'''-l''' ''walltime''=[[days:[hours:]]minutes:]seconds[.milliseconds]</tt> or by shortcuts <tt> -l walltime=[1w][1d][1h][1m][1s]</tt>. Shorter jobs have higher priority than longer jobs. '''Default value''' of this option is 24 hours. Examples: ** '''-l walltime=10:00''' &ndash; job will last at most 10 minutes ** '''-l walltime=3:00:00''' &ndash; job will last at most 3 hours ** '''-l walltime=1d1h ''' &ndash; job will last at most 1 day and 1 hour, i.e. 25 hours ** '''-l walltime=4w ''' &ndash; job will last at most 4 weeks, i.e. 28 days * ''number of machines and processors'' &ndash; is set by <tt>'''-l''' ''nodes''=[number of machines]:''ppn''=[number of proccessors]</tt>. '''Default value''' of this option is 1 machine with 1 processor. Examples: ** '''-l nodes=1:ppn=1''' &ndash; one processor at one machine ** '''-l nodes=1:ppn=16''' &ndash; one machine with 16 processors ** '''-l nodes=20:ppn=2''' &ndash; 20 two-processors machines <br />The job can be run on the machine with the '''same or greater''' amount of processors. If you are unsure about the number of needed processors, ask for an exclusive reservation of the whole machine using the parameter <tt><nowiki>#excl</nowiki></tt>. The best way is to use also the parameter for the number of machine's CPUs. E.g.: ** '''-l nodes=1:ppn=4:nodecpus4#excl''' * ''amount of needed memory'' &ndash; is set by '''-l''' ''mem''=[number][mb/gb]. '''Default value''' of this option is 400 MiB. Examples: ** '''-l nodes=1:ppn=1,mem=200mb''' &ndash; job needs 200 MiB ** '''-l nodes=1:ppn=8,mem=1gb''' &ndash; job needs one machine with 8 CPU and with 1GiB (1024MiB) memory. ** '''-l nodes=3:ppn=2,mem=2gb''' &ndash; job needs 3 machines, each with 2 CPU and 2GiB memory * ''properties of machines'' &ndash; are set in parametr <tt>-l</tt> in element <tt>nodes</tt> by [http://metavo.metacentrum.cz/en/docs/aplikace/run/pbs_long.html#vlastnosti properties] written after colon, e.g. ** '''-l nodes=1:ppn=1:x86_64''' &ndash; machine for run of 64bit applications (AMD64 or EMT64T) ** '''-l nodes=1:ppn=1:brno:amd64''' &ndash; machine in Brno with processor AMD Opteron ** '''-l nodes=1:ppn=4:nodecpus4:xeon''' &ndash; machine with 4 CPU Intel Xeon * ''licence'' &ndash; is set in individual parametru <tt>-l</tt> or after comma with specification of machines and memories by element called licence (see [http://metavo.metacentrum.cz/en/docs/aplikace/run/pbs_long.html#licence detailed tutorial]), e.g. ** '''-l nodes=1:ppn=1 -l mem=500mb -l matlab=1''' &ndash; job requires 1 licence MATLAB, 1 processor and 500MiB memory ** '''-l nodes=1:ppn=1,mem=500mb,matlab=1''' &ndash; the same, but written in a different way ** '''-l maple10=1''' &ndash; job requires 1 licence Maple version 10 ** '''-l maple11=1''' &ndash; job requires 1 licence Maple version 11 ** '''-l fluent=4''' &ndash; job requires 4 licences Fluent * ''amount of temporary [[Scratch storage|scratch space]]'' (fast and reliable) required for job processing; the storage space is identified by the '''<code>$SCRATCHDIR</code>''' environment variable. The amount and type of the required scratch space is determined by the <tt>-l scratch=size:type</tt> parameter, where '''size''' is a number and unit (e.g. 20gb), and '''type''' must be one of the following options: '''ssd''','''local''', '''shared''', '''first''' (''local'' is the default option). For the meaning of all the options, see the [[Scratch storage|pages describing the scratch storages]]. '''Default value''' of this option is 400 MiB. Examples of specification: ** '''-l scratch=1''' &ndash; requirement for 1 KiB of a local scratch space ** '''-l scratch=1mb''' &ndash; requirement for 1 MiB of a local scratch space ** '''-l scratch=1gb:local''' &ndash; requirement for 1 GiB of a local scratch space ** '''-l scratch=10gb:ssd''' &ndash; requirement for 10 GiB of a scratch space allocated on SSD storages ** '''-l scratch=100gb:shared''' &ndash; requirement for 100 GiB of a scratch space allocated on a shared network storage ** '''-l scratch=500gb:first''' &ndash; requirement for 500 GiB of a scratch space allocated just on the master nodes * ''sending the information emails about the job state'' For example: ** '''-m abe''' &ndash; sends an email when the job aborts (a), begins (b) and completes/ends (e) You can use the tool [http://metavo.metacentrum.cz/en/state/personal Command qsub refining] for condition definition. ==Advanced options== ===Queues=== You can specify requested queue as one the qsub options: qsub -q 'target queue' -l option1 ... Current list of '''available queues''' including their set up can be found in [http://metavo.metacentrum.cz/pbsmon2/queues/list;jsessionid=E1A073D8ABBFD72EAF2689B26F04597E Pbsmon application] (queues with limited access are marked with lock) or by '''qstat -q''' or '''qstat -Q''' commands inside your terminal. *Queue ''backfill'' is designed for "filling" jobs. It has a low priority, jobs can be killed while needed, only one machine is allowed for computing, but there can be lots of them. *Queue ''preemptible'' allows users to use owner reserved clusters where no common queues are available. Cluster owners are able to stop the other people's running jobs and run their own jobs immediately. Jobs can be suspended up to 30 days. ===Properties of computational nodes=== {{RedInfo|These options could actually limit you is some way, use them with caution!}} Provided list of attributes may not be complete. You can find actual list on the web in section [http://metavo.metacentrum.cz/en/state/props#prop2node Node properties] * '''geographical location''' &ndash; can influence the network latency: ** <tt>brno</tt> &ndash; machines in Brno (ÚVT MU Botanická, kampus MU Bohunice, FEKT VUT Kolejní,...) ** <tt>praha</tt> &ndash; CESNET server room, Zikova 4, Praha ([http://www.mapy.cz/#l=13&q=Zikova%204%2C%20Praha&qp=16.596626080632586_49.20844553354211_16.602945323445912_49.211947312330864&z=16&t=s&x=14.390699797258687&y=50.10180066042857 map]) ** <tt>plzen</tt> &ndash; ZČU server room, Univerzitní 22, Plzeň ([http://www.mapy.cz/#l=13&q=49°43'31.0008%22N%2013°21'2.9988%22E&qp=9.286047670246775_46.53665806533458_22.5066048883429_53.82105128356061&z=16&t=s&x=13.350825200425016&y=49.72527016025126 map]) ** <tt>budejovice</tt> &ndash; PřF JU server room, Branišovská 31, České Budějovice ([http://www.mapy.cz/#z=15&q=48°58'41.272%22N%2C%2014°26'46.196%22E&qp=14.246999_48.902117_14.563781_49.053439_11&t=s&x=14.449261&y=48.978087&d=coor_14.446166%2C48.978131_0_1&c=14-23-27-28-29-30-31 map]) ** <tt>feec</tt> &ndash; room E131, building FEKT VUT, Kolejní 4, Brno ([http://www.mapy.cz/#mm=ZP@sa=s@st=s@ssq=KolejnÃ4, Brno@sss=1@ssp=133095920_131999984_133110384_132012752@x=138066006@y=132939121@z=16 mapa]) ** <tt>uvtmu</tt> &ndash; ÚVT MU server room, Botanická 68a, Brno ([http://www.mapy.cz/#q=49°12'35.85%22N%2C%2016°35'59.202%22E&qp=16.592352_49.205320_16.608259_49.214456_15&z=15&t=s&x=16.602878&y=49.209853&d=coor_16.599778%2C49.209958_0_1&c=14-23-27-28-29-30-31 mapa]) * '''location in cluster''' &ndash; allows to run a job in within the scope one clusteru, eg: ** <tt>cl_mandos</tt> &ndash; machines from mandos cluster ** <tt>^cl_mandos</tt> &ndash; excludes the machines of mandos cluster * '''cpu type''' &ndash; sets the processor instruction set ** <tt>amd64</tt> &ndash; processors with AMD 64 architecture ** <tt>em64t</tt> &ndash; Intel Xeon processors with 64-bit extension ** <tt>x86_64</tt> &ndash; unification of amd64 em64t, ie. processors capable of running the 64-bit applications ** <tt>x86</tt> &ndash; unification of i386 a x86_64, ie. processors capable of running the 32-bit applications ** <tt>xeon</tt> &ndash; Intel Xeon processors ** <tt>opteron</tt> &ndash; AMD Opteron processors * '''number of CPU''' &ndash; suitable in case of need of whole cluster reservation: ** <tt>nodecpus1,nodecpus2,nodecpus4,nodecpus8,nodecpus16,nodecpus32</tt> &ndash; number of CPUs in node (1, 2, 4, 8, 16, 32). Finds the machine with accordingly required number of processors. * '''operating system''' ** <tt>linux</tt> &ndash; machines with Linux OS ** <tt>debian</tt> &ndash; machines with Linux OS, Debian distribution ** <tt>debian50,lenny</tt> &ndash; machines with Linux OS, distribution Debian version 5.0 "lenny" ** <tt>debian60</tt> &ndash; machines with Linux OS, distribution Debian version 6.0 * '''networking cards''' ** <tt>myrinet2000</tt> &ndash; Myrinet networking cards ** <tt>infiniband</tt> &ndash; Infiniband networking cards * '''for queue''' &ndash; don't set these attributes, they are chosen automatically according to requested queue: ** <tt>q_normal,q_short,long,q_preemptible,q_forpriviliged,...</tt> ** <tt>ncbr</tt> &ndash; machines of NCBR (National Center for Biomolecular Research) group ** <tt>iti</tt> &ndash; machines of ITI (Institut Teoretické Informatiky) project ** <tt>pa177</tt> &ndash; machines for parallel programming course PA177 of FI MU ** <tt>loslab</tt> &ndash; machines of Loschmidt Laboratories group * '''other''' ** <tt>nfs4</tt> &ndash; machines with NFSv4 availability, eg. with directory /storage ** <tt>data-kky</tt> &ndash; machines at ZČU with data directories of cybernetics faculty ** <tt>per,per2,per3,per4,per5,per4or5</tt> &ndash; more granular segmentation of perian cluster ** <tt>ssd</tt> &ndash; attribute of quark 14-16 machines ** <tt>tarkil1-1.cesnet.cz</tt> &ndash; the name of the machine is also a property, useful for the precise placement of the job === Memory requirements === To specify job memory requirements you can use 4 parameters to describe the resources: * '''mem''' - used amount of physical memory used at one requested node. * '''vmem''' - maximal amount of virtual memory used by all processes belonging to a job ('''v''' stands for '''v'''irtual memory) * '''pmem''' - maximal amount of virtual memory used by each of job processes ('''p''' stands for '''p'''rocess) * '''pvmem''' - maximal amount of virtual memory used by each of job processes Parameters ''pmem'' and ''pvmem'' '''reduces''' memory consumption for each process of the job apart (they are almost equivalent to following commands: <code>ulimit -m</code> and <code>ulimit -v</code>), they don't reserve mamory at all Parameter ''vmem'' reduces virtual memory of whole job but it also can't be also used for memory reservation. So we can forget that vmem, pmem and pvmem commands exist. Parameter which you definitely want use is '''mem'''. Using it you reserve and reduce certain amount of memory for each node requied for your task. For example: nodes=5:ppn=2,mem=200mb It request 5 machines and 2 processors and 200MB physical memory on each of them. The scheduler never use one physical machine as two or more nodes from job assingment although a machine with 4 or more processors is available only two prosecors are allocated to the job. In other words if you request <code>nodes=N</code> you'll really get N various machines even if it be possible to reach the same number of the processors using less machines. The scheduler use parameter '''mem''' when schedules jobs to machines. For example the scheduler can schedule one task which require 4GB per node to a dualcore machine with 4GB physical memory or it can schedule to this manehcine 2 tasks one of which requires 3GB per node and the other one requires 1GB per node, but it doesn't schedule to this machine a task which requires 5GB or if there's a task requires 3GB the scheduler won't schedule another task which require 2GB to this machine. Maximum for one specific machine can be founded by pbsnodes command: for example: $ pbsnodes skirit1.ics.muni.cz |grep resources_available.mem result: resources_available.mem = 1030960kb or on the page containing detail of a machine, look for '''memory - total'''. Unfortunately even if you use '''mem''' do not guarantee that the job get from the operating system whole required memory. The system can use some memory to cover indirect cost. Because of that asking for a little more memory than the job need is recomended. But do not ask for too much memory because it may limit the processors which are sufficient for this job. If you don't enter the '''mem''' parameter at all the default value will be used. The default value is quite low and it may neddlessly restricts the job. Because of that always enter this parameter. [[Category:Grid services]] [[Category:Scheduling system]] a196avbqu0gle2s2vshmfskyj6c8rhb 0 580 5297 5296 2016-05-16T18:36:28Z 127.0.0.1 /* usage of the iti queue */ wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[How to compute/Requesting resources|Requesting resources]]|}} {{BoxPoložka|[[How to compute]]|}} {{BoxPoložka|[[About scheduling system]]|}} }} This topic shows examples of job submitting. First, you might want to see [[How to compute]] and [[:Category:Scheduling system|Scheduling system]] to understand these examples. == How to request resources example == ==== 1 processor on 1 node ==== You want to reserve one processor at one node for your job (this is also default setting): <source lang="bash"> nodes=1:ppn=1 </source> ==== 3 nodes in Pilsen ==== {{RedInfo|You can limit yourself when you specify the location of nodes!}} You we want to reserve 2 nodes in Brno, 3 nodes in Pilsen and 4 nodes in Prague, 1 processor on each node: <source lang="bash"> nodes=3:ppn=1:plzen </source> ==== 5 nodes with 2 cpu for 100 hours and 4gb of RAM memory ==== You want to reserve 5 nodes with two processors each for 100 real time hours, each process requires 200 MB of virtual memory: <source lang="bash"> nodes=5:ppn=2,walltime=100:00:00,mem=4gb </source> == Examples of usage == {{Box|Related topics {{BoxPoložka|[[How to compute/Job management|Job management]]|}} }} ====1 CPU job ==== Create file '''test.sh''' in your favourite text editor (''vim'', ''pico'' ...) <source lang="bash"> #!/bin/sh echo this is just a test script </source> Submit the job: <source lang="bash"> qsub -q short -l nodes=1:ppn=1 test.sh </source> qsub returns a line similar to '''<number>.arien.ics.muni.cz'''. The number is '''jobID''', which you can use for e.g. [[How to compute/Job management|tracking job status]]. When the job is finished, you will find two files in the directory, which you submitted the job from &ndash; ''test.sh.o<jobID>'' and ''<test.sh.e<jobID>''. One represents standard output and the other one standard error output. ==== multiprocessor task ==== Multiprocessor tasks usually use commercial software which is already prepared to use more processors, see following exaple of [[Gaussian-GaussView application|gaussian task]]. Despite of this fact you can create a multiprocessor task on you own. You must specify the queue and the number of processors while submitting the job, for example: queue for a job which last at maximum 2 hours and 3 machines, 2 processors on each qsub -q short -l nodes=3:ppn=2 soubor.sh Your requests can be also specified in executing file <code>file.sh</code> using lines which start with <code><nowiki>#PBS</nowiki></code>, for example: <nowiki> #!/bin/sh #PBS -N myJobName #PBS -q short #PBS -l nodes=3:ppn=2 #PBS -j oe #PBS -m e # # describtion from 'man qsub': # -N ... declares a name for the job. The name specified may be up to and including 15 characters in length. It # must consist of printable, non white space characters with the first character alphabetic. # -q ... defines the destination of the job (queue) # -l ... defines the resources that are required by the job # -j oe ... standard error stream of the job will be merged with the standard output stream # -m e ... mail is sent when the job terminates </nowiki> and then job can be executed by typing qsub soubor.sh After that the script is executed on one allocated machines. List of allocated machines can be found in the file name of which is stored in system variable <code>$PBS_NODEFILE</code>. Another procecces can be executed on another allocated machined by <code>pbsdsh</code> command. (For more details look at man pages: <code>man qsub</code> and <code>man pbsdsh</code>.) For example: <nowiki> #!/bin/sh #PBS -l nodes=3:ppn=2 #PBS -j oe #PBS -m e # #in file name of which can be found in variable PBS_NODEFILE, is list of allocated machines echo '***PBS_NODEFILE***START*******' cat $PBS_NODEFILE echo '***PBS_NODEFILE***END*********' # spusti dany prikaz na vsech pridelenych strojich pbsdsh -- uname -a </nowiki> Submit, wait until job's complete (status C) a check throug output: $ '''qsub file.sh''' 272154.skirit-f.ics.muni.cz $ '''qstat 272154.skirit-f.ics.muni.cz'''<nowiki> Job id Name User Time Use S Queue ---------------- ---------------- ---------------- -------- - ----- 272154.skirit-f file.sh makub 00:00:00 R short $ </nowiki>'''qstat 272154.skirit-f.ics.muni.cz'''<nowiki> Job id Name User Time Use S Queue ---------------- ---------------- ---------------- -------- - ----- 272154.skirit-f file.sh makub 00:00:00 C short $ </nowiki>'''cat soubor.sh.o272154'''<nowiki> ***PBS_NODEFILE***START******* skirit4.ics.muni.cz skirit7.ics.muni.cz skirit8.ics.muni.cz skirit4.ics.muni.cz skirit7.ics.muni.cz skirit8.ics.muni.cz ***PBS_NODEFILE***END********* Linux skirit4.ics.muni.cz 2.6.17.7 #1 SMP Thu Aug 3 11:16:56 CEST 2006 i686 GNU/Linux Linux skirit8.ics.muni.cz 2.6.17.7 #1 SMP Thu Aug 3 11:16:56 CEST 2006 i686 GNU/Linux Linux skirit4.ics.muni.cz 2.6.17.7 #1 SMP Thu Aug 3 11:16:56 CEST 2006 i686 GNU/Linux Linux skirit7.ics.muni.cz 2.6.17.7 #1 SMP Thu Aug 3 11:16:56 CEST 2006 i686 GNU/Linux Linux skirit8.ics.muni.cz 2.6.17.7 #1 SMP Thu Aug 3 11:16:56 CEST 2006 i686 GNU/Linux Linux skirit7.ics.muni.cz 2.6.17.7 #1 SMP Thu Aug 3 11:16:56 CEST 2006 i686 GNU/Linux </nowiki> Observe that the request <code>nodes=3:ppn=2</code> was handled by allocation of machines skirit4, skirit7 a skirit8. Each of them appeared twice because two processors had been requested. ==== Gaussian ==== Next example is showing how to run Gaussian 98 program. We demonstrate transfer of input and output data, possibility to enter parameters for PBS directly in the script, sending mail when the job is finished, work with modules and running the job in the scratch volume. The job is submitted into long queue and should not run longer than 240 hours. skript run <nowiki> #!/bin/sh # Script for running g98 via PBS: # # Notification concerning job finish or job cancel by mail: #PBS -m ae # # Inicialization of module and adding module g98: . /packages/run/modules-2.0/init/sh module add g98 # # The preparation of job's input data: if anything fails, the job # terminates with a non-zero exit code: cp /storage/brno1/home/ruda/computations/xxx.com $SCRATCHDIR || exit 1 # # Change of working directory: if it is not successful, the job will be finished # with non-zero return code: cd $SCRATCHDIR || exit 2 # # Job running: g98 xxx.com # # Copying-out the job's output data cp xxx.log /storage/brno1/home/ruda/computations/ # # Cleaning the scratch directory rm -rf $SCRATCHDIR # # That is all, end of script "run".</nowiki> Running: qsub -q long -l nodes=1:ppn=1,cput=240:00:00 run {{TODO|Missing MPI and interactive examples}} <!--==== MPI job ==== {{Box|Related topics {{BoxPoložka|[[How to compute/Parallelization|Parallelization]]|}} }} This example shows how to run an MPI job. Program /home/ruda/tmp/cpi.gm is compiled using MPI from module mpich-gm and we will run it at 4 CPUs (2 nodes) with myrinet card in Pilsen. skript run-gm <nowiki> #!/bin/sh # Inicialization of module and adding module mpich-gm . /packages/run/modules-2.0/init/sh module add modules ics.muni.cz module add mpich-gm # # Job running mpiexec -comm mpich-gm cpi.gm # # That is all, end of script "run".</nowiki> Running: qsub -q normal -l nodes=2:ppn=2:plzen:myrinet run-gm ==== Interactive job ==== {{Box|Related topics {{BoxPoložka|[[How to compute/Interactive jobs|Interactive jobs]]|}} }} In this example we will run an interactive job using 2 nodes. <source lang="bash"> qsub -I -N mojeUloha123 -q short -l nodes=2:ppn=1 qsub: waiting for job 255242.skirit-f.ics.muni.cz to start qsub: job 255242.skirit-f.ics.muni.cz ready </source> At this point a new terminal is opened -> your interactive job has begun. <source lang="bash"> makub@skirit15$ echo $LOGNAME makub makub@skirit15$ ... some work ... makub@skirit15$ exit </source> qsub: job 255242.skirit-f.ics.muni.cz completed --> [[Category:Grid services]] [[Category:Scheduling system]] fq3au5uph631e1g5zut60dil8ez3jg3 0 584 2033 2032 2016-05-08T11:19:25Z Root 1 /* Limited resources */ wikitext text/x-wiki ''Cgroups'' (shortcut of Control Groups) is a feature of Linux which allows to control usage of resources (memory, I/O operations, networking etc.) for certain group of processes. See [https://en.wikipedia.org/wiki/Cgroups Wikipedia] for more information. Cgroups is offered by MetaCentrum since October 2015 in order to offer special settings of user jobs - you can set an impassable resources limits of your job. {{YellowInfo|Read more about qsub options [[How to compute/Requesting resources|here]].}} '''Default settings''' is not to use Cgroups, you have to request it with qsub option '''-­W cgroup=true''': [[Image:cgroups.png]] You won't definitely affect other users with Cgroups in your job, because you cannot exceed given resources. === Machines supporting Cgroups === There is only limited number of machines which support Cgroups; your job is computed on one of them, if you use the '''-­W cgroup=true''' option. You can find list of these machines [http://metavo.metacentrum.cz/pbsmon2/props#cgroup here] (look for the ''cgroup'' property). === Limited resources === Cgroups limits just two offered resources by MetaCentrum - processors and physical memory. * '''number of CPUs''' - Cgroups checks if it is used at maximum N processors per one machines, when using '''qsub -l nodes=X:ppn=N''' * '''physical memory''' - Cgroups checks if requested memory is not exceeded, e.g. when using '''qsub -l mem=1gb''' {{GreenBox|Physical memory limitation applies just to the physical memory, not to a virtual memory. Process can still allocate some additional memory on disk in so called ''swap''. We plan to change implementation that option '''mem''' works with sum of RAM and the swap.}} [[Category:Scheduling system]] 0djs23okzvfczphkwgemtlvqzkbptxx 0 585 5265 2034 2016-05-14T09:15:55Z Root 1 wikitext text/x-wiki ScaleMP is a software layer (hypervisor), which connects multiple ordinary x86 servers into one SMP machine. The goal of this aggregation is to connect resources (CPU, RAM) from many servers into one virtual machine. This method is practically opposite to standard virtualization where one server runs multiple virtual machines. Company ScaleMP defines this architecture as ''versatile SMP'' or vSMP a its implementation as vSMP Foundation, widely known as ScaleMP. ==Who is ScaleMP good for== '''Advantages of ScaleMP''' * Your job has all aggregated memory and CPUs available - you don't have to use manual parallelization * The system can offer significant savings in SW licences which are bound to to psychical nodes (although legal interpretation could be discutable) '''Disadvantages of ScaleMP''' * Generally, an application using ScaleMP has slower computation speed than the application with manual parallelization * You have to pay some effort to prepare your job, if you would like to achieve its solid computation speed Generally, the system is particularly ''suitable for self-running jobs'' that require a large number of processors and/or memory and also jobs which parallelization is difficult and you have to pay some nontrivial effort for its preparation. The system is ''not ideal for running multiple'' smaller jobs. ==What is the machine made of== Machine [http://metavo.metacentrum.cz/pbsmon2/hardware#alfrid.meta.zcu.cz alfrid] is made of 16 servers (IBM x3550 M4) connected with 2xFDR infiniband (SX6036), which have 256 cores and 4TB memory in sum. As [[Scratch storage|scratch storage]] is used disk array IBM V3700 (24x 10k 600 gb disks, connected with SAS into masternode). Cluster has 2x10gb/s uplink (from masternode) into MetaCentrum network located at ZČU. The system contains also the 17th server, which is used as license and image server of ScaleMP a also contains local Ethernet switch (SG500-52) for IPMI aggregation and network booting. ==How to access the machine== You have to be member of MetaCentrum (see [[Application]]). Users from groups ''iti'' or ''kky'' have automatic access; for individual access contact HW owners via email to meta@cesnet.cz and also send copy to honzas@ntis.zcu.cz. If you are affiliated to science centre NTIS, ask for membership representatives of noticed groups (KKY - Jan Švec|Flídr Miroslav, KMA - Jan Nejedlý, KFY - Jiří Houška, KIV - Ladislav Pešička, KME - Miroslav Horák), who can add new users to groups ([[Group_managers's_manual|tutorial is here]]). You can check your membership in a ''/etc/group'' file on any MetaCentrum machines or see MetaCentrum webpage [http://metavo.metacentrum.cz/cs/myaccount/personal My account -> Personal data], in the '''VO settings''' tab select '''VO MetaCentrum''' and in the down right section of the page should appear list of groups. ==How to run a job on the machine== First of all, you should read [[How to compute]] topic, if you are new in MetaCetrum. Jobs use the machine ''alfrid'' via special queue '''scalemp''' and you have to specify concrete group, which you use for access. A job submit might look like: qsub -q scalemp -W group_list=iti ..... In case you need the whole machine, request all CPUs ('''-l nodes=1:ppn=256''') or request the machine exclusively ('''-l nodes=1#excl'''). You can also use [[How to compute/Requesting resources|qsub option]] '''-I''' for [[How to compute/Interactive jobs|interactive job]]. {{YellowInfo|You can login to the machine ''afrid'' via ''ssh'', but it is necessary to use the scheduling system for computing}} ==How to take full advantage of the machine in a job== Applications using ScaleMP are struggling with speed of the computation and because of that, it's important to focus on location of processed data and try to do optimization steps. There is available ScaleMP documentation on the [http://www.scalemp.com/support/docs/execution-guidelines/ ScaleMP webpage] (after registration), where you can find e.g. [http://www.scalemp.com/wp-content/uploads/2013/04/Best_Practices_for_running_applications_with_vSMP_Foundation.pdf Best practices] or examples for successful running the most used SW (Ansys, Matlab, Fluent, VASP ...) Other suggests, examples and tutorial focusing on optimization own applications are available in form of examples on the machine ''alfrid'':/opt/ScaleMP/examples We found these steps as effective: # read at least basic ScaleMP documentation # have a look, if there is not a detailed tutorial for your SW # prepare a basic job, which will run about 1 hour # try to run the job on an empty machine with suggests from this topic # evaluate the achieved job speed acceleration against standard HW # if you are not satisfied, try to contact our [[:Category:User support|user support]] (meta@cesnet.cz) or ScaleMP support (Support@ScaleMP.com) === How difficult is to edit an application - model example: the basic edit === First optimization attempt can be done even before running an application - add the following lines to your startup (batch) script: .... # find $NCPU CPUs, which are the most suitable CPULIST=`/opt/ScaleMP/numabind/bin/numabind --offset $NCPU --flags=best 2>/dev/null` # connect this process and his children with these concrete CPUs taskset -pc $CPULIST $$ # run the computation .... It is probably possible to achieve additional enhancement with help of other optimization settings or libraries offered by company ScaleMP, see links to documentation higher. === How difficult is to edit an application - model example: multiply two matrices in MATLAB === A simple MATLAB script, which multiply two random matrices: nproc=str2num(getenv('NPROC')); maxNumCompThreads(nproc); % Enable multithreading setenv('MKL_NUM_THREADS', getenv('NPROC')); % Set parameters numRuns = 1; % Number of runs to average over % dataSize = 100000; % Data size to test dataSize = str2num(getenv('DATASIZE')); x=rand(dataSize,dataSize); % Random square matrix % Matrix multiplication for i = 1:numRuns y=x * x; end quit For ScaleMP is specific to fix processes to concrete processors before running a computation: #! /bin/bash # Make MATLAB accessible export PATH=$PATH:/software/matlab-8.5/bin # # Get arguments export NPROC=16 [ "_$1" != "_" ] && NPROC=$1 export DATASIZE=10000 [ "_$2" != "_" ] && DATASIZE=$2 # # ScaleMP specific stuff # # Intel OpenMP and MKL environment variables export KMP_AFFINITY=compact,granularity=fine export KMP_LIBRARY=turnaround export MKL_DYNAMIC=FALSE export MKL_VSMP=1 # # Find which CPUs to run MATLAB on CPULIST=`/opt/ScaleMP/numabind/bin/numabind --offset $NPROC --flags=best 2>/dev/null` export NUMBIND_RESERVED=$CPULIST taskset -pc $CPULIST $$ # # Run MATLAB echo "Running with $NPROC threads on $DATASIZE x $DATASIZE matrices" matlab -nodisplay -nojvm -r matmul 2>&1 [[Category:Scheduling system]] 3j7l26bn5cxpbu8l4oxmk2g6un45akq 0 473 2097 2066 2016-05-12T12:37:24Z Root 1 wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[:Category:Policies|Policies]]|}} }} The membership in MetaCentrum is allowed without any restrictions only to persons from academic environment of the Czech republic with research objectives. In a case of interest from commercial company (i.e. its research part), it is necessary to consult your interest and its possible fulfilment. As the MetaCentrum user you are allowed to use currently available computational resources for your research aims and projects. As the MetaCentrum employee your membership brings you access to resources, information and materials necessary for your work. By submitting an [http://metavo.metacentrum.cz/en/application/index.html application] you also accept the following rules which had to be checked in [[MetaCentrum end user statement]] before application submission. These rules are obligatory and they should be observed as their violation can lead to end your MetaCentrum membership. * [[Usage rules/Security rules|Security rules]] * [[Usage rules/Operation rules|Operation rules]] * [[Usage rules/Acknowledgement|Appreciation formula / acknowledgement]] * [[Usage rules/Account|Account validity]] Once a year, usually at the end of calendar year, the user asks about [http://metavo.metacentrum.cz/en/myaccount/prodlouzeni account extension]. This prolonging application is accompanied by MetaCentrum usage description, description of achieved results and a list of publications for the last year. Using of MetaCentrum VO is free of charge, we only require the users to [[Usage rules/Acknowledgement|acknowledgement]] MetaCentrum in their publications. [[Category:Policies]] sg0deajuwz5csrw3705nr7vmcf4gohf 0 523 1518 1441 2016-03-07T09:34:59Z Root 1 wikitext text/x-wiki Secure and effective utilization of MetaCentrum resources is possible only if there will be some operational rules fulfilled. Study therefore really carefully the security rules for work in MetaCentrum and in the network generally. Helpful is also the article [[How to compute/Accessing machines|Accessing machines]]. =MetaCentrum security rules= * User has to protect his/her account by a non-trivial [[Kerberos authentication system|Kerberos]] password. The user can not tell this password to anybody and has to protect the password by all available means - that except other things means do not use it in open (non-encrypted) protocols as is e.g. standard unic (non-Kerberized) telnet or ftp. * Non-trival password is such that can not be deduced (even after simple mutations) from the data known about account owner and that is not a name of a person, animal or object (even after simple mutations). The administrators are authorized to perform tests to look for non-trivial passwords; in the case of positive results will inform the account owner by secure way. Followingly the account owner have to change the password immediately (the original account can be locked until the time of password change). * Users can not borrow accounts among each other. If it is necessary to share the account due to some reasons (e.g. software installation), it has to be done using Kerberos (i.e. through ".k5login" file) and it should be also discussed with specific administrators. * The password can not be accessible at non-encrypted form (e.g. saved to disc). If the user writes down the password to a paper, it can not be together with login name and the user has to take care this information properly as other secret information (personal papers, credit cards and so on). If the password is revealed or lost it is necessary to change as soon as possible via [http://metavo.metacentrum.cz/en/myaccount/heslo.html My Account - Password change] and especially inform MetaCentrum administrators (from the same reasons as during credit card loss -- the user is responsible for his/her account and all activities at the account or realized from the account). * Making MetaCentrum accounts accessible through <tt>.rhosts</tt> is forbidden. We do not recommend usage of this authentication method even at other machines. If this method is used, it is necessary to take into account lower security of specific account. Users should not used this way of authetication for their own good. * Usage of non-secured protocols telnet, ftp etc. at other machines is not recommended as well. If this authentication method is used, one has to take into account lower security level of specific accout (and disclosure of the password in open form transferred by the network). It is forbidden use such account to log in to MetaCentrum through Kerberos or SSH protocol due to danger of security breach using weakly secured account followed by usage of fake ssh or fake Kerberos with a trojan horse and consequently danger of breach of MetaCentrum accounts. * Strange behaviour of your account, appearance or disapparance of files, unknown processes running under your identity and so on has to be necessarily reported to the administrators of MetaCentrum. [[Category:Policies]] td5v79j6psnfvcjl760ur8nc4od23x1 0 524 1520 1519 2016-03-07T09:37:35Z Root 1 wikitext text/x-wiki * Users have to use MetaCentrum resources adequately with respect to other users. This practically means not to over-load disc capacities and to submit all demanding calculations using the PBS batch system. * [[Scratch storage|Scratch volumes]] at MetaCentrum are dedicated for saving temporary data until the job is finished. These volumes are definitely not intended for data backup - the violation of this rule can cause a disciplinary action; moreover the culprit can lose his/her data because volumes /scratch* are not backed up! * AFS volumes are intended for software installations. * NFSv4 volumes are meant as unlimited storage space. The default quota of 5TB is just a protection from an unintended filling up the whole space, it can be enlarged any time. * Home directories are mapped into /storage to the nearest disk array. * For long-term archivation, the subdirectories in /storage ending with '''-archive''' or '''-hsm''' are intended. The method of archivation is selected by the name of subdirectory in users's directory, e.g. ''VO_metacentrum-tape_tape'' means storing on two magnetic tapes. '''IMPORTANT NOTICES:''' * Jobs that wil be running more than several minutes MUST NOT run at any MetaCentrum computer outside the load balancing system (PBS)! In other word, ALL computational processes have to be inserted into specific job queue. * There are special folders dedicated for saving the temporary data during computations - the folder identified by the <code>$SCRATCHDIR</code> system variable, which points to a scratch location dedicated for a job. These are the fastest discs in the system therefore it is necessary to move the jobs results to your home directory or to /storage. * Volumes /scratch*. are not backed-up! [[Category:Policies]] 1zllca7vuogorsx2gklbyn07av0zveh 0 525 1521 1443 2016-03-07T09:37:46Z Root 1 wikitext text/x-wiki The user of MetaCentrum is obliged to use the following acknowledgement formula in all your publications created with the support of MetaCentrum. Please fill citations of your publications [https://metavo.metacentrum.cz/cs/myaccount/pubs in our system], you will get privileged access to all resources of MetaCentrum or CERIT/SC centre as a bonus. ==Text of the official acknowledgement formula (EN version)== Text of the MetaCentrum official acknowledgement formula: {{Rámeček-green|text=Access to computing and storage facilities owned by parties and projects contributing to the National Grid Infrastructure MetaCentrum provided under the programme "Projects of Projects of Large Research, Development, and Innovations Infrastructures" (CESNET LM2015042), is greatly appreciated.}} Text of the CERIT-SC official acknowledgement formula (zewura, zebra, zegox, zigur, zapat, zefron,...; ungu, urga): ): {{Rámeček-green|text=Access to the CERIT-SC computing and storage facilities provided by the CERIT-SC Center, provided under the programme "Projects of Projects of Large Research, Development, and Innovations Infrastructures" (CERIT Scientific Cloud LM2015085), is greatly appreciated.}} Text of the combined MetaCentrum/CERIT-SC official acknowledgement formula: {{Rámeček-green|text=Computational resources were provided by the CESNET LM2015042 and the CERIT Scientific Cloud LM2015085, provided under the programme "Projects of Projects of Large Research, Development, and Innovations Infrastructures"}} [[Category:Policies]] q52xl67rlto5nk154019n75ot5u83cn 0 526 1522 1444 2016-03-07T09:37:57Z Root 1 wikitext text/x-wiki Once a year, usualy at the end of calendar year, the user asks about [http://metavo.metacentrum.cz/en/myaccount/prodlouzeni account prolonging]. This prolonging application is accompanied by MetaCentrum usage description, description of achieved results and a [https://metavo.metacentrum.cz/cs/myaccount/pubs list of publications] for the last year. Unacomplishing this procedure leads to blocking of a user account, but it still can be prolonged. Four years without prolongation means deleting a user account and so unavailability of any possible stored user data. [[Category:Policies]] px1g134jw0ww66btg8xa0fyijxh92xf 0 521 1523 1436 2016-03-07T09:38:19Z Root 1 wikitext text/x-wiki Hereby I certify that I have read and understood the [[Usage rules | terms and conditions]] of use of MetaCentrum hardware and software equipment and documentation. I will adhere to the rules and regulations stated below and to directions of system administrators of respective MetaCentrum nodes. I am aware that system and application software installed on MetaCentrum nodes are subjects to their respective license agreements. I declare I will acquaint myself with software license terms, conditions, and regulations before first use of the software and I will adhere to these. I am aware that the computer equipment and most of the software equipment was purchased solely for academic use with exclusion of all commecial use. I declare I will adhere to the following rules when working with MetaCentrum equipment and equipment of each supercopmuter center: *Account on MetaCentrum equipment is strictly personal and cannot be lent or otherwise transferred to other individuals. Account must be protected with non-trivial password that must not be disclosed to other individuals nor stored on any accessible place. *It is forbidden to perform any activity that could lead to tampering of property rights of owners and authors of available software equipment, especially to copy copy-protected software, to analyze or reverse engineer software protected by a license disallowing to do so, or to run illegitimately obtained software. *Any activity leading to priviledge escalation above assigned level (access rights, disk space, computer time) on MetaCentrum equipment is strictly forbidden. Especially it is forbidden to attemt to decrypt passwords of other users, to misuse the defects of systems or software to acquire unassigned priviledges. Any activity that could result in obtaining access priviledges by unauthorized person or otherwise jeopardize operation of MetaCentrum resources is forbidden. *Activites that could adversely affect work of other users (e.g. running of jobs with unusually large demands on processor time or computer memory, storage of large amount of data in public directories, overloading of network subsystem, etc.) must be consulted with system administrators in advance and must not be run without prior notice. *Confidential data and software of users must be protected by setting proper access rights or using encryption. MetaCentrum takes no responsibility for misuse of data and software that are property of users. *Utilization of MetaCentrum hardware and software equipment must be acknowledged by users in particular publications (scientific articles, technical reports etc.). Copy of these publications must be made available to MetaCentrum representatives on demand. *Access right is granted for one calendar year. At the end of the year I will deliver short written information about the activities performed and obtained results. MetaCentrum reserves the right to use this information in annual reports and similar publications. I am aware that violation of these rules might lead to account constriction and eventual consequences by law. Furthermore I declare that if I learn about violation of these rules I will immediately inform system administrators of MetaCentrum. I agree with the fact that my personal information will be processed and stored by CESNET, z.s.p.o. for accounting purposes. The above mentioned information will be available only to administrators of computational resources for purpose of identification of users in extent necessary for operation of MetaCentrum according to the law No. 101/2000 (jurisdiction of the Czech Republic). [[Category:Policies]] dn5g01m3skfa2rgm56x38ssesht422m 0 586 2067 2016-05-09T17:13:33Z Root 1 Založena nová stránka s textem „This topic explains concepts and procedures for resolving a security incident in MetaCentrum. ==Concepts== * Security incident & ndash ; suspicious behavio…“ wikitext text/x-wiki This topic explains concepts and procedures for resolving a security incident in MetaCentrum. ==Concepts== * Security incident & ndash ; suspicious behaviour of a service or a machine offered by MetaCentrum, administrator or user account abusing * Security group & ndash ; a group of MetaCentrum administrators which supervises RT system and responds to all reports of a potential attack. Its main goal is coordination during an incident and preparing final report. ==Duties== Administrator of a service or a machine offered by MetaCentrum '''has to''' report every incident which might potentially affect another server, machine or user identity in MetaCentrum. Administrator of a service or a machine offered by MetaCentrum '''has to''' respond to every incident which contains his local resources, services or user identities in relation with MetaCentrum. Security group '''has to''' react to every initiative of an incident which is reported to RT system. ==Communication== All communication has to be done via email address [mailto:meta@cesnet.cz meta@cesnet.cz]. An email creates new ticket in MetaCentrum RT system. Copy of information sent to CSIRT teams has to be also sent to the email adress. It would be great if you could send compressed source code of malfunction software as well. ==Responsibility== Responsible persons during an incident are: administrator of machine, which was attacked and security team of MetaCentrum. The administrator has to provide all necessary information to security team via email (see ''Communication'' above). The security team has to prepare final report of the incident. ==Data protection== The email address mentioned in ''Communication'' above is read just by a few MetaCentrum administrators and delivered emails are not public. The RT system is authenticated. In the case of an exchange of very sensitive data (passwords, private user data) it is necessary to use a secure channel, which will be selected after consultation with the MetaCentrum security group (again through RT system). ==Procedure at detecting an incident== #Detecting and report via RT system #Initial analysis of the incident #Restriction of further incident spreading #A detailed analysis of origin of the incident #Recovery from the incident #Lessons from the incident #Final report [[Category:Policies]] jrw401eqbdpmldcvbdtyb044rivaahe 0 76 268 267 2016-01-23T15:48:45Z Root 1 naimportována 1 revize wikitext text/x-wiki __NOTOC__ ===Overview=== [http://perun.metacentrum.cz/web/index.shtml Basic information about Perun.] Link following to Perun {{PerunGUILink|rel=vo GUI}} === First steps === New as manager? Start with our manuals! * [[VO managers's manual|VO managers's manual]] * [[Group managers's manual|Group managers's manual]] * [[Facility managers's manual|Facility managers's manual]] === Tutorials === Lost in Perun system? Our step-by-step tutorials help you. * [[Use cases|Tutorials and use cases]] === Advanced tutorials === Do you want dig deep in Perun system? Interesting facts for you. * [[Registration as member of VO| How to create an application form - long version]] * [[Perun CLI]] * [[What I can do as 'ROLE']] * [[Perun remote interfaces|Perun remote interfaces]] * [[Perun external sources|Perun external sources]] * [[Synchronization of groups with external source|Synchronization of groups with external source]] === The Perun project === Do you really want to know how Perun works? * [[Conceptual scheme and definition of terms]] * [[Perun overall schema]] * [[Perun component schema]] * [[Perun enrollment schema]] * [[Propagation]] * [[Namespaces]] * [[What can Perun control]] === Technical documentation === * [[Known bugs]] {{info|If your issue is not covered by our manual, please report your problem to [mailto:perun@cesnet.cz perun@cesnet.cz]}} [[Category:Perun]] [[Category:For managers]] [[Category:English]] skennbtr0rm52drhd07aa3tnilg3c94 0 77 270 269 2016-01-23T15:48:46Z Root 1 naimportována 1 revize wikitext text/x-wiki {{Perun-en-template}} ==[[Conceptual_scheme_and_definition_of_terms#Groups_-_Groups|Group]] administration == All VO members can be inserted into groups. Every group gets an access rights to resources in its VO. Groups can be nested like a tree and the access rights are inherited in the same way. Number of groups and subgroups is not limited. VO member can get rights to administer group. Group name must be unique in the VO. Groups can be managed via CLI or GUI. Please do not forget that particular VO must be selected before manipulation with group. Group administration via {{PerunGUILink|rel=vo/list GUI}} is done by clicking on ''Groups'' [[File:group.png|frameless|alt=groups|]] in left menu. The list of groups included in VO will appear. Button ''Create'' creates new group, button ''Remove'' removes selected groups (but only if you are also VO manager). By clicking group name in the list of groups, detailed information about group and list of parent group appears. Subgroups will be shown by clicking ''Subgroups'' [[File:Group_add.png|frameless|alt=subgroups|]] button, subgroup will be created by ''Create'' and removed by ''Remove''. == Creating group manager from VO member == Members of VO can get the right to manage group in VO and become group manager. Group manager can add or remove group members, create subgroups and assign them managers. Manager doesn't have to be member of VO. Number of managers is not limited. Administration of managers can be also done by clicking button ''Managers'' [[File:administrator.png|frameless|alt=managers|]] in left menu in {{PerunGUILink|rel=vo/list GUI}}. New manager will be added by ''Add'' button and removed by ''Remove'' button. == Adding VO members into group == Both VO manager and Group manager can add new members in a group. Member who want to be added in the group must be also member of VO containing the group. One member VO can be member of more than one group; therefore has an access to all resources available in all groups. Because of a hierarchy of groups in the system, user must be member of parental group before he will be added in subgroup. Oppositely, Group manager must be member of parental VO, but he might be member of group. === Step by step tutorials === * [[Add member to group]] * [[Remove member from group]] == Managing an application form == [[Create_application_form|Tutorial to create basic application form in VO/group.]] [[Approve application to VO|Tutorial to approve application form in VO/group.]] == More tasks == [[My_role_is_GROUP_MANAGER|All taks are listed here.]] [[Category:Perun]] [[Category:For managers]] [[Category:English]] sj4el7bscscwqxkzxabe7z9ec4yqg1c 0 78 272 271 2016-01-23T15:48:46Z Root 1 naimportována 1 revize wikitext text/x-wiki {{Template:Perun-en-template}} [[Conceptual_scheme_and_definition_of_terms#Virtual_organization_-_VOs|VO]] manager is a person or group of people allowed to manipulate with VO data. First manager of particular VO must be created by Perun admin.Vo manager can utilize both CLI interface and {{PerunGUILink|rel= GUI web GUI}} to operate system entities. VO manager is responsible for following tasks: == Adding users among [[Conceptual_scheme_and_definition_of_terms#Users_into_VO_-_Members|VO members]] == VO manager is allowed to select user that will become a member of the VO from [[Conceptual_scheme_and_definition_of_terms#External_identity_resources_-_Ext_sources|external identity resources]] or resources available to the VO. Choosing your VO from the {{PerunGUILink|rel=vo/list VO list}} in GUI, you will see not only details about VO but also menu items in left part. VO manager uses menu to administer VO. Button ''Members'' [[File:user_green.png|frameless|alt=VO|]] serves to list all VO members. Button ''Add'' enables to select new user from list of candidates. Button ''Search'' is used to search for the user. User detail is shown by clicking the user in the list of members. Button ''Remove'' serves to remove member from VO. == Creating a VO manager from VO member == VO manager is allowed to create a VO manager from any other VO member. Detailed tutorial [[Add_VO_manager|how to add VO manager]]. == [[Conceptual_scheme_and_definition_of_terms#Groups_-_Groups|Group]] administration == VO inner structure can be enhanced by hierarchy of groups. Every group has an access rights to particular resources in VO. Group administration is described in a detail in [[Group_administrator%27s_manual#Groups_administration|Group manager's manual]]. == Creating group manager from VO member == VO manager can create a group manager from VO member. How to create a Group manager is described in [[Group_administrator%27s_manual#Creating group manager from VO member|Group manager's manual]]. == Adding VO members into group == VO manager and Group manager can add a VO member into group. Whole process is described in the [[Group_administrator%27s_manual#Adding VO members into group|Group manager's manual]]. == Creating resource tags == VO manager can create tags for resources. Detailed tutorial: [[Create resource tags|how to create resource tag]]. ==[[Conceptual_scheme_and_definition_of_terms#Resources_-_Resources|Resource]] utilization == Resource is part of Facility that can VO utilize. It is created by Facility manager, but only VO manager is allowed to manipulate with it. First, select particular VO in {{PerunGUILink|rel=vo/list GUI}}. Button ''Resources'' [[File:server_group.png|frameless|alt=resource|]] shows list of resources in VO. Button ''Remove'' removes selected Resources. By clicking the row with name of Resource, the detailed information appears. By clicking the tab ''Assigned groups'', the list of assigned groups is listed. Tab ''Assigned services'' shows list of assigned services. By clicking on the button ''Service settings'', page with attributes' settings is shown. These attributes are necessary to proper functionality of services. How to: * [[Add member to resource]] * [[Remove member from resource]] * [[Assign group to resource]] * [[Assign service to resource]] * [[Assign tag to resource]] == Managing an application form == [[Create_application_form|Tutorial to create basic application form.]] [[Approve application to VO|Approve application to VO.]] == Creating rules to account extensions == It is necessary to set attribute ''membershipExpirationRules'' for VO. Attribute can be added in ''Settings'' in VO. Its items can be: * '''doNotAllowLoa''' - list of [[Conceptual_scheme_and_definition_of_terms#Level_of_assurance|LoAs]] separated by comma, which won't be allowed in VO (users can't become members). * '''period''' - time period to extend membership. It can be set as fixed date (without year), e.g. 1.1 or as number of days/months/years with prefix "+" that defines time period that extends membership. Units are d = day, m = month, y = year, e.g. +128d extends account to 128 days. +6m, + 1y. * '''doNotExtendLoa''' - list of [[Conceptual_scheme_and_definition_of_terms#Level_of_assurance|LoAs]] separated by comma, that are not extensible. * '''gracePeriod''' - when present date of initial application or extending request equels extension date minus ''gracePeriod'' then user account is extended to the next time period (''period'' date in next year). Value is in format number days/months/years. Units are d = day, m = month, y = year, e.g. 128d, 6m, 1y * '''periodLoa''' - an exception in period for given LoA. Format of value is: LoA|period[.]. ''LoA'' is given Loa number and ''period'' is in same format as a ''period''. Optional dot at the end means whether extend an account to user with filled ''membershipExpiration'' or not. If dot is present, user with filled membershipExpiration is not allowed to extend an account. == More tasks == [[My_role_is_VO_MANAGER|Full list of all VO manager tasks]] [[Category:Perun]] [[Category:For managers]] [[Category:English]] obhsw69j0lpnqde989oxliv9xdll0xl 0 79 274 273 2016-01-23T15:48:46Z Root 1 naimportována 1 revize wikitext text/x-wiki {{Template:Perun-en-template}} == Basic use cases in Perun == ===For VO manager=== * [[Approve application to VO]] * [[Add VO manager]] * [[Create group in VO]] * [[Remove member from VO]] * [[Remove group from VO]] * [[Remove VO manager]] ===For Group or VO manager=== * [[Add member to group]] * [[Remove member from group]] == Advanced use cases in Perun == [[Less frequent tasks|Advanced use cases in Perun]] [[Kategorie:Perun]] [[Kategorie:English]] [[Kategorie:Tutorials]] ge6r9b75c23vemuhf6kw01gig0jhta9 0 80 276 275 2016-01-23T15:48:46Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Use cases|Back to Use cases]] ===For VO manager=== * [[Add member to resource]] * [[Add member to VO without application]] * [[Assign group to resource]] ** [[Assign group to resource#Assign group to resource by using tags|Assign group to resource by using tags]] * [[Assign service to resource]] * [[Create service member]] * [[Create application form]] * [[Remove member from resource]] * [[Set VO attributes]] * [[Set group attributes]] * [[Set service attributes]] * [[Create resource tags]] * [[Assign tag to resource]] === For Group manager === * [[Set group attributes]] === For Facility manager === * [[Set service attributes]] === For members === * [[Set member attributes]] === Overview of roles' cooperation === * [[Configure a service]] * [[Configure_a_services_-_use_cases|Configure a service -- ssh keys root]] [[Kategorie:Perun]] [[Kategorie:English]] i7h62ocrf44wl18cybx4fx2iqfjjmin 0 81 278 277 2016-01-23T15:48:46Z Root 1 naimportována 1 revize wikitext text/x-wiki {{Template:Perun-en-template}} == What I can do as ''ROLE'' == * [[My role is VO MANAGER|as VO MANAGER]] * [[My role is GROUP MANAGER|as GROUP MANAGER]] * [[My role is FACILITY MANAGER|as FACILITY MANAGER]] * [[My role is SELF|as SELF]] [[Kategorie:Perun]] [[Kategorie:English]] [[Kategorie:For managers]] hvn8u722z80mlyfod7q8t8nbb8e46jy 0 82 280 279 2016-01-23T15:48:47Z Root 1 naimportována 1 revize wikitext text/x-wiki As VO manager I can do: * [[Approve application to VO]] * [[Remove member from VO]] * [[Create group in VO]] * [[Remove group from VO]] * [[Add VO manager]] * [[Remove VO manager]] * [[Add member to VO without application]] * [[Add member to resource]] * [[Remove member from resource]] * [[Create service member]] * [[Create application form]] * [[Assign group to resource]] * [[Assign service to resource]] * [[Assign tag to resource]] * [[Set VO attributes]] * [[Set service attributes]] * And together with ''GROUP MANAGER'' ** [[Add member to group]] ** [[Remove member from group]] ** [[Set group attributes]] [[Kategorie:Perun]] [[Kategorie:English]] [[Kategorie:For managers]] 1hq8cpf35qaazarj2aquxu4yxvxy25b 0 83 282 281 2016-01-23T15:48:47Z Root 1 naimportována 1 revize wikitext text/x-wiki [[What I can do as 'ROLE']] === My role is ''GROUP MANAGER'' === * [[Add member to group]] * [[Remove member from group]] * [[Set group attributes]] [[Kategorie:Perun]] [[Kategorie:English]] [[Kategorie:For managers]] bn02wphwfniwrro12hq6aqfgerha84w 0 84 284 283 2016-01-23T15:48:47Z Root 1 naimportována 1 revize wikitext text/x-wiki {{Perun-en-template}} As Facility manager, I can do: * [[Set service attributes#FACILITY MANAGER role needed|Set service attributes]] [[Kategorie:Perun]] [[Kategorie:English]] [[Kategorie:For managers]] 8ziu4slvnqc3prmb4eue48fwspdpc8t 0 85 286 285 2016-01-23T15:48:47Z Root 1 naimportována 1 revize wikitext text/x-wiki [[What I can do as 'ROLE']] === My role is ''SELF'' === * [[Set member attributes]] [[Kategorie:Perun]] [[Kategorie:English]] [[Kategorie:For managers]] hnjwhilldf30jsos7vlw5ltf8nvmd10 0 86 288 287 2016-01-23T15:48:47Z Root 1 naimportována 1 revize wikitext text/x-wiki {{Template:Perun-en-template}} Perun is a wide system providing user management and user-connected services to the various types of facilities (single machines, clusters, storage, elements or even software licenses) in various infrastructure sizes (from managing single access to software license to creating accounts in cloud-like environment with thousands CPUs). Whole Perun is based on Virtual organizations, groups and facilities. Managers of these basic entities cooperate in Perun to gain desired results. All of them use complex GUI (or CLI) to manage his part of duties. In short, the facility manager provides resources to VOs and configures settings of these resources or the facility itself. '''Key feature:''' As was already mentioned, facility manager just provides the resource to the VO but does not manage users who utilize his facility. Since facility manager is familiar with VO's user policy and every single VO member must agree with policies, facility manager has assurance that all users utilizing his facility fulfill his requirements. The VO manager manages members of his VO and decides which groups under the VO can utilize resources provided to the VO by the facility manager. Moreover groups within the VO can obtain even a self-management by having their own group manager. He can add and remove members in group and edit an application form. Application form is highly customizable tool to manage users coming to the VO (or the group). Content and required information is fully in the hands of particular manager. Also the way how to accept filled forms is up to managers, it could be done manually or automatically. '''Key feature:''' Application form is a great way to make user management easier. User in application form fills all data which manager needs and by submitting application form user agrees with terms of use and (defined by VO manager). Groups often covers work-life roles. It is recommended to create groups matching with roles (for example groups managers, user support, maintenance) consequently is really easy to list, add or remove some resource accessible to the whole group. By adding new member into group covering his work role, all resources and accesses he needs to have to do his job is available for him. Another use case is when VO manager adds a new member into group, the member gains all accesses available to group's members. '''To sum up''', all users' settings are in one place and easy to manage via GUI or CLI. There is a great level of customization in nearly every aspect of user management. Even more, system is prepared to fit to nearly any customer's requirement (for example, new service, new attribute etc.). Also many processes in system can be done automatically, or not (depending on managers' will). All together creates flexible and scalable role-based system of sharing resources. == Concept scheme definition == [[File:Concept_scheme.png|frameless|500px|alt=Concept_scheme.|Concept_scheme.]] === Virtual organization (VO) management === VO has members and managers. VO is very basic unit to assign resources. * [[Conceptual_scheme_and_definition_of_terms#Application_form|Application form management.]] * [[Conceptual_scheme_and_definition_of_terms#Users_into_VO_-_Members|VO member management.]] * [[Conceptual_scheme_and_definition_of_terms#Groups_-_Groups|Group management within the VO.]] === Resources management within VO === [[Conceptual_scheme_and_definition_of_terms#Resources_-_Resources|Resources]] are provided to VO based on agreement. There is no formal form for this agreement, whole process depends on mutual agreement of VO manager and facility manager. * [[Conceptual_scheme_and_definition_of_terms#Perun_services_-_Services|Service management]] Service management defines which type of resources is provided and assign these resources to VO that consequently use them. * Management of configuration for services Services in resources must be configured before user can access any of them. Perun pushes configuration data to resources according to their configuration. == Basic terms description == === '' An account shared among users -- Service member'' === Service member is an account that is not based on physical person's identity, but created in the system and one or more users can be assigned to operate with it. Except this functionality, it works as a normal user with all his rights. Every service member account has an evidence: * it is for service purposes * who are allowed to manipulate with it (change password, change an email) For example: Regular backuping of system must be done, but it is unsafe to store backups into personal accounts. Moreover, there are more people responsible for backuping and backups would be spread into several personal accounts. Service member solves this task because all backups are stored in his storage space and all responsible people have an access to it. For example: If an application (Hudson in our case) needs an access to system via username and password, then is unsafe to provide somebody's personal account, so service member account is advisable to provide. Tutorial to [[Create_service_member|create service member]]. === ''Application form'' === Application form serves as a gate to the particular VO or group. It is form created by administrator of VO or group for users. After application form is filled and approved, user becomes a member of particular VO. Approval process can be automatic or manual (approved by VO manager). New user must be verified by IdP or information system of organization. === ''Attributes - Attributes'' === Entities in Perun (e.g. facility, resource, service) and relations (member-resource, user-facility) can contain more information in a form of attributes. Attributes serve as support information to the services. ''Example:'' Facility manager defines list of available shells in cluster as an attribute. VO manager can select a shell from this list for his members. This list is saved as an attribute at resource. Every single member can have preferred shell that is saved as an attribute in the relation user-facility. Attributes can store anything that it is necessary to propagate to the device. Also it could be set by user. Attributes have type string, int, array of string. === ''Evidence of facility - Hosts'' === Hosts serve as an evidence where facility is placed. It is only an evidence, as a destination of services is taken information from [[Conceptual_scheme_and_definition_of_terms#Service goals - Destinations|Destinations]]. === ''External identity resources - Ext_sources'' === External identity resource is assigned to each user. This resource can be identity providers, some LDAPs, SQL database of organizations, outputs of information systems ... etc. and primary user identity is taken from it. External identity resources serve as a proof that user is a part of the real organization and therefore can access to the resources and services. External resources can serve as a condition to become a member of virtual organization or a group in the VO. Information from external resource can be used to automatic export of user into VO. '''LoA''' (level of assurance) is a term connected with external resources, specially ''federations'. Loa should be same for all external identity providers, see [[#Level of assurance| Level of assurance]] ===''Facility owners - Owners'' === Contact to facility manager. It serves to VO manager to contact him to create an agreement to gain an access to resources. === ''Groups - Groups'' === Groups can be created within the VO automatically based on some user property or manually based on VO manager decision. Groups serves to refine internal structure to access to resources. Groups and subgroups of members who are part of the VO. Every VO member is part of group "Members" by default. === ''Perun services - Services'' === Perun services propagate data from Perun database to the facility (clusters, data storage.. etc.). Services create configuration files in clusters (e.g. /etc/passwd), fill Kerberos, actualize list of acceptable certificates etc. Services run automatically based on predefined events in Perun system (e.g. change in membership in a group, new member in VO...). Propagation can be run also manually. === ''Resources - Resources'' === Resource is "bundle of services" provided by owner of physical device or software, e.g. one storage volume with its size and access protocols, possibility to submit jobs in specific queues (or utilize specific clusters), possibility to use specific software. Resource is one basic unit provided to VO on the basis of oral or written agreement. Resources are used by members. VO manager sets access rights for members in VO by utilization of group system. Agreement between VO manager and facility manager sets "maximal scope of use or limiting conditions" that can not be exceed by members in VO. ==== ''Purpose of resource - Resource tags'' ==== To resouces can be assigned tags which marked special properties of resources. (For example tag ''computational node'' means, this resource is used for users to compute their tasks, tag ''service machine'' marked resource as machine for internal operations at VO etc..) === ''Service goals - Destinations'' === Many services create configuration files and propagate them to some goal. This goal can be device, some special device, email address or URL where is output of service. === ''Types of resources - Facilities'' === ''Note'': Facility is historic name and now is not exactly matching with the meaning. Facility is an entity (pc, software, storage) existing in a real world. To gain an access to facility, [[Conceptual_scheme_and_definition_of_terms#Resources_-_Resources|resource]] must be created. Following conditions facility meets: * they are using same technology and share configuration (e.g. they are NFSv4 volumes, they need an account in the local machines.. etc., Perun pushes its data coherently) * they have single management (one manager responsible for all pieces can be identified - e.g. data storage elements owned by CESNET). On the other side, "service provided to user", e.g. NFSv4 volumes export from one physical storage to the other should be consider as one facility, not more. Attributes are connected with facilities. === ''Users - Users'' === User is an account in Perun system that matches with physical person (it could happen that physical person has more accounts but it is not recommended). At least one external identity is connected with user. === ''Users into VO - Members'' === Member is the user in the particular VO. Relation between user and VO can contain attributes. === ''Virtual organization - VOs'' === VO has members and managers. VO is very basic unit to assign resources. VO is organization of users who have common specific goals and want to be managed by VO manager. VO could have specific requirements to access to resources, to verify the user etc. Possible specification is defined in VO policy. == User account states== * '''invalid''' User is a VO member, but some important information to propagate the services is missing. When all attributes are complete, user can be switched to state '''valid'''. * '''valid''' User is VO member and all necessary attributes are filled. * '''suspended''' User move to '''suspended''' state after security incident connected with his account * '''expired''' Membership in VO has expired, to move to state '''valid''' user must apply for an extension. * '''disabled''' An access to all resources has been blocked to the member. == Level of assurance == * '''Level 1''' email address is verified * '''Level 2''' identity and institution is verified * '''Level 3''' level 1+ strict requirements to password quality and its use [[Category:Perun]] [[Category:English]] ooztyskh0dilsbdakbwk7uju3j3pi50 0 87 290 289 2016-01-23T15:48:48Z Root 1 naimportována 1 revize wikitext text/x-wiki {{Template:Perun-en-template}} Only Perun admin is competent to work with facility in a full scope, but Facility manager is entitled to do following tasks: == Add [[Conceptual_scheme_and_definition_of_terms#Facility_owners_-_Owners|owner]] == Every facility must have his owner to know what organization is responsible for its utilization. There are list of owners in the page of selected facility in {{PerunGUILink|rel=fac/list GUI}}. Click ''Owners'' [[File:User_silhouette.png|frameless|alt=owner|]] in left menu. Buttons ''Add'' and ''Remove'' serve to add new owners or remove the old ones. == Create a [[Conceptual_scheme_and_definition_of_terms#Resources_-_Resources|Resource]] == Resource is a part of facility that serves for particular VO. Only Facility manager is allowed to create and delete a resource. Facility manager can only see which resources are connected to facility by selecting particular facility and clicking on ''Resources'' [[File:server_group.png|frameless|alt=resources|]]in the left menu. In the same place can be resource created or deleted. == Assign [[Conceptual_scheme_and_definition_of_terms#Perun_services_-_Services|service]] to [[Conceptual_scheme_and_definition_of_terms#Resources_-_Resources|Resource]] == Services serve to synchronize state of facility (in real world) with its state configured in the Perun system. Assigning service to resource, facility manager allows VO connected with resource to utilize this service. Services are assigned to each resource separately because not every services are required by every VO. Service must be set on facility first, then on the resource. [[Assign_service_to_resource|Tutorial how to assign services]] == Set [[Conceptual_scheme_and_definition_of_terms#Attributes_-_Attributes|attributes]] == It is essential to set all necessary attributes for proper utilization of facility. Attributes must be set in a good order. First attributes in the facility, then attributes in the resource. In addition, attributes in resources can be set only in scope of attributes in facility. Every service has required attributes that must be set for correct work of service. [[Set_service_attributes|Tutorial to set service attributes]] == Show [[Conceptual_scheme_and_definition_of_terms#Service_goals_-_Destinations|destinations]] == Destinations are tightly connected with services. Only facility manager can assign services to destinations. Service destinations are available after clicking on button ''Service destinations'' in left menu of particular facility. There are several types of Destinations: * host - data will be sent to the machine using SSH * url - data will be sent to the URL using POST * email - data will be sent in the body of email to the given address * semail - data will in the body of digitally signed email == How to monitor states of services== *{{PerunGUILink|rel=fac/list GUI:}} select facility and click on ''Propagation status'' in left menu. Detail of service is obtained by clicking on the line with service. * CLI: [[Perun_CLI#getTaskResultsForDestinations|getTaskResultsForDestinations]] * Perl: $propagationStatsReaderAgent->getTaskResultsForDestinations(destinations => \@destinations) All states of all facilities administered by you, you see by clicking ''All Facility states '' [[File:Arrow_right.png|frameless|alt=facility]] in the left menu. = Services from the facility manager's point = ==== Introduction ==== [[Conceptual_scheme_and_definition_of_terms#Perun_services_-_Services|Services]] are tools to keep real-life facilities synchronized with their settings in the Perun system. Most of services is realized by set of scripts: * '''''gen''''' scripts serves to select data from database in Perun * '''''send''''' scripts serves to send data from database to destinations (real-life facility) * '''''slave''''' scripts serves to operate with data in the facilities. Facility and VO manager make an arrangement for services that facility utilize and for structure and type of data. Especially '''''slave''''' scripts are important for facility admin. ==== Service propagation ==== By clicking on the ''Service propagation'' [[File:Arrow_right.png|frameless|alt=facility]] in the left menu, list of services and their propagation status is obtained. By button ''Force propagation'', propagation of particular service is queued. Dependig on the lenght of queue, it may take several seconds to propagate your task. ==== ''Slave'' scripts instalation ==== There are packages containing ''slave'' scripts for debian and RPM systems, prepared for facility managers. In a case of need, packages for other systems can be prepared. Scripts to gen, send and slave are available from Meta repository (package perun-slave). Perun is authorized to machine using SSH key. All scripts in the destination facility are available in ''/opt/perun/bin/''. For facilities which are installed by puppet modules is prepared module described at https://forge.puppetlabs.com/ceritsc/perun ===== How to install scripts to facility ===== * Into file<code><nowiki>/etc/apt/sources.list</nowiki></code> add line: deb ftp://depot1.mc.cesnet.cz/ all main * How to work with [[Instalace#Debian package repository|Debian package repository]] * Packages ''perun-slave'', ''perun-slave-meta'' and ''perun-slave-meta-key'' must be installed to debian. apt-get install perun-slave perun-slave-meta perun-slave-meta-key SSH key without use ''perun-slave-meta-key'' can be inserted manually: insert into <code><nowiki>/root/.ssh/authorizes_keys</nowiki></code> * For RH based machines is package available in RPM repository: ftp://homeproj.cesnet.cz/rpm/perunv3/ New edits of ''slave'' scripts are made concerned matching versions of ''gen'' scripts. Version number contains 3 digits: * first is version of system Perun (now 3) * second is changed when ''gen'' script changed and new version of slave scripts is necessary to install * third is changed when ''gen'' script is changed, but slave script accept it There is strong tendency to keep ''slave'' scripts same in all facilities, but sometimes facilities have specific requirements (e.g. settings of home directories, paths to scripts etc.). As a consequence, '''''pre''''' and '''''post''''' scripts are introduced. They differ in every facility and complement functionality of ''slave'' scripts. When ''slave'' script starts, it searchs for <code><nowiki>pre_</nowiki></code> script in the service directory and runs it before its own functionality starts. The same process is after ''slave'' script execution, the <code><nowiki>post_</nowiki></code> scripts are run. All scripts are executed in the alphabetical order; therefore we recommend to name scripts according to pattern: pre_XX_name, where XX is number with two digits that specify order of script's execution. '''''Pre''''' and '''''post''''' scripts are created by facility managers. In addition, there is no need to create these scripts if facility manager is satisfied with default settings. Finally, there is a freedom in utilizing services thanks to customizable '''''pre''''' and '''''post''''' scripts. Perun is installed to the machine (facility) with all possible slave scripts, but blacklisting and whitelisting of scripts depend on configuration in the Perun system. As far as whitelisting is concerned, no other service with an exception of the whitelisted ones will be executed. As for blacklisting, all services will be executed except of the blacklisted one. Example of blacklisting '''''/etc/perunv3.conf''''': SERVICE_BLACKLIST[0]=fs_scratch SERVICE_BLACKLIST[1]=fs_home Example of whitelisting'''''/etc/perunv3.conf''''': SERVICE_WHITELIST[0]=passwd SERVICE_WHITELIST[1]=group ===== Pushed synchronous propagation ===== When installation is complete, it is necessary to propagate services. Command''remctl'' is available in package remctl-client in distributions. remctl perun.ics.muni.cz perun propagate Propagate all services defined in the machine. [[Category:Perun]] [[Category:For managers]] [[Category:English]] li6jrf0ajxtytotjfkgfs9adk2aj246 0 88 292 291 2016-01-23T15:48:48Z Root 1 naimportována 1 revize wikitext text/x-wiki {{Perun-en-template}} == Introduction == [[Conceptual_scheme_and_definition_of_terms#Application_form|Short introduction to application form]] == Application types== There are two types of application form initial and for extension. Both can be customized in one GUI under ''Settings'' button according to needs of VO/group manager. * '''Initial''' application serves to first contact with VO/group * '''Extension''' serves to extend your already existing account in VO/group. There are two ways to accept application form * '''Manual''' - when user submits his application form, email notification to administrator is sent, admin can accept or reject in GUI, it is default option * '''[[Registration_as_member_of_VO#Automatic_approval|Automatic]]''' - when user submits his application form, application is accepted automatically == Creating a form == As VO or group manager, select your VO/group and in the left menu click on ''Application form'' (in list of advanced setting). If it is first time, you create an application form, note "Form doesn`t exists, do you wish to create new one?" is shown. Click on ''Create''. There are two ways to fill a form with content. * '''Copy it.''' Button ''Copy from group'' or ''Copy from VO'' serves to copy application form from other groups/other VOs to what you have an access. Do not forget to [[Create_application_form#Form_editation|edit copied form]] before you actually use it. :) * '''Do it on your own'''. Fill form by items [[Create_application_form#Creating_an_item|creating them one by one]]. == Form editation == Each item can be '''edited, moved or removed.''' * ''Edit'' returns you to the window of creating an item. * ''Move'' by clicking on arrows you can move an item one step up or one step down in a form * ''Delete'' removes an item from form {{mbox|If you close form without saving, all unsaved changes are lost.}} == Creating an item == Button ''Add'' serves to add new item to the form. First you need to fill these pieces of information: * '''Short name''' serves as short name, or is displayed in the form if no language label is inserted. * '''[[Registration_as_member_of_VO#Item_types|Type]]''' serves to select one of the several types of item from drop down menu. * '''Insert after''' is to specify the place after which is new item inserted After clicking on ''Create'' button, menu with more options appears. * '''Short name''' -- same as short name from previous window * '''Input widget''' -- same as '''[[Registration_as_member_of_VO#Item_types|Type]]''' from previous window * '''Display on application''' -- item will be displayed on '''initial form, extension or both''' (tick what you want) * '''Required''' -- when checked, item is required to fill and marked by asterisk (*) * '''Destination attribute''' -- attibute '''into which value''' of item will be '''stored''' * '''Federation attribute''' -- attribute which value can be '''prefilled''' with '''data from user's federation''' * '''Regular expression''' -- '''regexp''' that validates item value Other tabs are the language ones. For '''each item''' and '''each language''' fields: * '''Label''': is used instead of short name of attribute * '''Help''': help for each item * '''Error''': plain text of error, which appear when item is required but not filled, or does not match with regular expression. It should solve as information for user what is wrong. Click on the ''Save'' [[File:disk.png|frameless|alt=VO|]] to save an item. To see the preview, use '''Preview''' button. All changes are only temporary, to save whole form, press '''Save'''. [[File:disk.png|frameless|alt=VO|]] {{mbox|If you close form without saving, the all changes made in form are lost.}} == E-mail notification [[File:email.png|frameless|alt=VO|]]== Button ''Notification'' [[File:email.png|frameless|alt=VO|]] serves to create an email notifications to users about application form. There are several [[Registration_as_member_of_VO#Notification_type|types]] of notification. *'''Created/user''' -- mail that will be sent to user after his registration to VO/group *'''Mail validation/user''' -- mail that will be sent to user to validate his email provided in application form *'''Created/manager''' -- notification to manager that new user wants to be a member of VO/group *'''Approved/user''' -- mail to user that his application was approved by manager *'''Rejected/user''' -- mail to user that his application was rejected by user *'''Invitation/user''' -- mail to user invitation to join this VO/group There are two '''[[Registration_as_member_of_VO#Language|language version]]''' (czech and english) to fill text of '''each notifications'''. For each type of application form (initial, extension) you have to have separate notification. Many tags serves as variables enabling insert system values in notification text, for example: {voName} inserts name of your VO. Whole list is '''[[Registration_as_member_of_VO#Dynamic_content_substitution|here]]''' or mentioned in GUI. === Good examples of email notification === *'''[[Notifications_--_good_examples#Created.2Fuser_--_initial|Created/user -- initial]]''' *'''[[Notifications_--_good_examples#Mail_validation.2Fuser_--_initial|Mail validation/user -- initial]]''' *'''[[Notifications_--_good_examples#Created.2Fmanager_--_initial|Created/manager -- initial]]''' *'''[[Notifications_--_good_examples#Approved.2Fuser_--_initial|Approved/user -- initial]]''' *'''[[Notifications_--_good_examples#Rejected.2Fuser_--_initial|Rejected/user -- initial]]''' *'''[[Notifications_--_good_examples#Created.2Fuser_--_initial|Created/user -- extension]]''' *'''[[Notifications_--_good_examples#Mail_validation.2Fuser_--_initial|Mail validation/user -- extension]]''' *'''[[Notifications_--_good_examples#Created.2Fmanager_--_initial|Created/manager -- extension]]''' *'''[[Notifications_--_good_examples#Approved.2Fuser_--_initial|Approved/user -- extension]]''' *'''[[Notifications_--_good_examples#Rejected.2Fuser_--_initial|Rejected/user -- extension]]''' == Example of application form == The purpose of this section is to give a simple schedule to help new manager to design new application form with all important items without deep knowledge. All application form should contain these items: * '''Introduction item''' with type [[Registration_as_member_of_VO#Item_types|Custom HTML text]] to explain some basic information about your VO/Group, contact to you ...etc. * '''Personal information''' section (do not forget to structure your application form for better understanding) ** '''Name''' - it can be visibly taken from federation (item with type: [[Registration_as_member_of_VO#Item_types|Input text field pre-filled from external source]], destination attribute ''displayName'' and federation attribute ''"cn"'') or manually inserted (item with type [[Registration_as_member_of_VO#Item_types|Input text multi-line field]]) ** '''Email''' - it should be obtained from federation hidden (item with type: [[Registration_as_member_of_VO#Item_types|Hidden input text pre-filled from external source]], destination attribute ''mail'', federation attribute ''mail'') ** '''Preffered email''' - it must be type [[Registration_as_member_of_VO#Item_types|Input text field for email]], after submitting the form, email with link to validate email is sent to preffered email and user must validate it by clicking on the link. (destination attribute mail, federation attribute mail) ** '''organization''' - value can be taken from federation, (destination attribute organization, federation attribute ''"o"'') ** '''prefferedLanguage''' - [[Registration_as_member_of_VO#Item_types|Single value selection from list]] with ''czech'' or ''english'' (destination attribute ''prefferedLanguage'') * '''Login and password''' section ** '''username''' -- type [[Registration_as_member_of_VO#Item_types|Input text field for username]], destination attribute ''login-namespace:(your_namespace)'' ** '''password''' -- type [[Registration_as_member_of_VO#Item_types|Input text field for passowrd]], destination attribute ''login-namespace:(your_namespace)'' * '''Details of membership''' section ** '''reason why user wants to be members of your group''', type [[Registration_as_member_of_VO#Item_types|Input text multi-line field]], destination attribute ''registrationNote'' ** '''research group''' where user works ** in case of extension form, '''reason of extension''' should be provided as [[Registration_as_member_of_VO#Item_types|Input text multi-line field]] item * '''submit button''', type [[Registration_as_member_of_VO#Item_types|Custom submit button]] Do not forget to save it. [[File:disk.png|frameless|alt=VO|]] {{info|How application form works in detail is described in the page [[Registration_as_member_of_VO|Create an application form -- long version]]}} [[Category:Application form]] [[Category:Tutorials]] [[Category:Perun]] [[Category:English]] 7xnwktztzyijnhhpblvm0quaexfs1xh 0 89 294 293 2016-01-23T15:48:48Z Root 1 naimportována 1 revize wikitext text/x-wiki Variables, for example: '''{voName}''' is automatically converted in text of notification into particular value. Your example '''{voName}''' has in text of notification VO Metacentrum value "VO MetaCentrum." ==Created/user -- initial== Subject: Confimation of application submission for VO '''{voName}''' ''Dear user, ''thank you for your application. The information you submitted has been successfully received. Your application will be reviewed by a VO '''{voName}''' manager.'' ''Please note that if you entered a new contact email (different from the e-mail received from your identity provider)'','' you will receive a verification e-mail. In that case the application will be passed to the VO manager after the e-mail is verified by you.'' ''Name: '''{displayName} ''' ''Login: '''{login-einfra}''' ''Application ID: '''{appId}''' ''Application state can be checked in "Applications" section: '''{appGuiUrl} ''You will be notified by another mail once your application approved or rejected. ==Mail validation/user -- initial== Subject: '''{voName}''': Email address verification ''Dear user, ''you have entered this email address as preferred contact on application in VO '''{voName}'''. ''Please verify your email address by clicking on link below. Until that, your application can't be approved.'' '''{validationLink}''' ''Yours sincerely '''{voName}''' ==Created/manager -- initial== Subject: New application for VO '''{voName}''' created ''Dear manager, ''new application for VO '''{voName}''' was created under ID='''{appId}''' by user: '''{displayName}''', '''{actor}''' / '''{extSource}'''. ''Application detail with all data submitted by user you can approve / reject application on '''{appDetailUrlKerb}''' ''If there was any error during application creation, information follows: '''{errors}''' ==Approved/user -- initial== Subject: '''{voName}''': Application no. '''{appId}''' approved ''Dear user, ''Your application for VO '''{voName}''' under ID='''{appId}''' was approved by VO manager.'' ''Name: '''{displayName} ''' ''Login: '''{login-einfra}''' ''Membership expiration date: '''{membershipExpiration}''' ''Administrative interface of Perun system, where you can change you contact details and manage different settings, can be found at:'' ''* FEDERATION (login and password from your institution or Hostel IdP):'''{perunGuiUrlFed}''' ''Yours sincerely '''{voName}''' ==Rejected/user -- initial== Subject: Application for VO '''{voName}''' rejected. ''Dear user, ''your application for membership in VO '''{voName}''' under ID='''{appId}''' was rejected by VO manager. Reason (if attached by manager) follows: '''{customMessage}''' ''Yours sincerely '''{voName}''' ==Created/user -- extension== Subject: Confimation of application submission for VO '''{voName}''' account extension ''Dear user, ''thank you for your application for VO ''' {voName}''' account extension. ''You will be notified by another mail once your application approved or rejected. ''Your '''{voName}''' ==Mail validation/user -- extension== Send only when user submitted new email address during extesion. Subject: '''{voName}''': Email address verification ''Dear user, ''You have entered this email address as preferred contact on application for membership in VO '''{voName}'''. Please verify your email address ''by clicking on link below. Until that, your application can't be approved. '''{validationLink}''' ''Your '''{voName}''' ==Created/manager -- extension== Subject: New application (account extension) for VO {voName} created ''Dear manager, ''new application for VO '''{voName}''' account extension was created under ID='''{appId}''' by user: '''{displayName}''', '''{actor}''' / '''{extSource}'''. ''Application detail with all user submitted data where you can approve / reject application on '''{appDetailUrlKerb}''' ''Your '''{voName}''' ''If there was any error during application creation, information follows: '''{errors}''' ==Approved/user -- extension== Subject: {voName}: Application no. {appId} approved ''Dear user, ''Your application ID='''{appId}''' for VO '''{voName}'' account extension was approved by VO manager. ''Name: '''{displayName}''' ''Membership expiration date: '''{membershipExpiration}''' ''Your '''{voName}''' ==Rejected/user -- extension== Subject: Application for VO '''{voName}''' account extension was rejected ''Dear user, ''your appliction for VO '''{voName}''' account extensin with ID='''{appId}''' was rejected by VO manager. Reason (if attached by manager) follows: '''{customMessage}''' ''Your '''{voName}''' [[Category:Application form]] [[Category:Perun]] [[Category:English]] bj72695vmey3rcvfpno9nfgl0umygy7 0 90 296 295 2016-01-23T15:48:48Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Use cases|Back to Use cases]] [[Less frequent tasks|Back to Less frequent tasks]] {{info|Typical example of this basic task is * one (or more) users are supposed to have root access to the resource * one (or more) users are supposed to have access to the particular machine/cluster/storage/software Inner structure of system isn't constructed for assigning an individual member to a single resource. As far as computing accounts are concerned, all VO members can use all resources in scope of the VO. To refine the resource utilization and to distinguish VO members requirements, VO members can be incorporated into (one or more) groups. Groups work as subunits of VO and VO resources can be assigned to the group depending on services that these groups utilize. As a consequence of adding a member into specific group, member gain an access to all resources assigned to the group. Not only resources, but also services could be assigned to the group. As the result, group of members with specific access rights to resources can be created.}} <hr> === Add member to resource tutorial === [[My role is VO MANAGER|''VO MANAGER]] ''role needed'' * visit {{PerunGUILink|rel=vo/list perunGUI VOs pages}} * use '''Select VO''' [[File:Building.png|frameless|alt=VO|]] from the left menu to display a list of VOs which you can manage * click on the row with a VO name to select the VO into which you want to add a member * select option '''Overview''' * in the area ''Quick tools'' click on the button '''Add member to resource''' [[File:add.png|frameless|alt=VO|]] * fill member's name or logname into a searching area and press the button [[File:find.png|frameless|alt=VO|]] '''Search''' * select a member by ticking the check-box and press the button [[File:add.png|frameless|alt=VO|]] '''Add''' * click on the button '''Continue''' [[File:arrow_right.png|frameless|alt=VO|]] * small frame for a resource selection is displayed ** you can select resource according to the facility where resource is located ** ..................OR................... ** you can select resource directly ** in both cases services assigned to the resource are displayed ** press the '''Continue''' [[File:arrow_right.png|frameless|alt=VO|]] button * a list of groups assigned to resource is displayed ** you can select from displayed groups by ticking the check-box and click on the button [[File:add.png|frameless|alt=VO|]] '''Add to group''' ** ..................OR................... ** if no group suits your requirements, you can create new one by pressing button [[File:add.png|frameless|alt=VO|]] '''Create group''' ** frame for creation of group is displayed and you have to fill: *** ''Name'' for group name *** ''Description'' for purpose of group creation *** check-box ''As sub-group'' for creating group as sub-group, in that case the menu ''Parent group'' is displayed to select parent group for newly created group ** click on the [[File:add.png|frameless|alt=VO|]] '''Create''' button to create new group ** (click on the [[File:stop.png|frameless|alt=VO|]] '''Cancel''' if you don;t want create new group) ** now, you can select newly created group by ticking the check-box and click on the button [[File:add.png|frameless|alt=VO|]] '''Add to group''' * confirmation window is displayed with list of resources where the member will be assigned * in case that list of resources (where group and your selected member is assigned) is suitable press the button [[File:accept.png|frameless|alt=VO|]] '''OK''' - group is assigned to all listed resources * ...................OR................... * if it is not suitable click on the button [[File:stop.png|frameless|alt=VO|]] '''Cancel''' and it returns you back to selection of group [[Category:Tutorials]] [[Category:Perun]] [[Category:English]] 0iid203r3a6lzbbfizd1u6lr4py56d2 0 91 298 297 2016-01-23T15:48:48Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Use cases|Back to Use cases]] == Approve application to VO == [[My role is VO MANAGER|''VO MANAGER]] ''role needed'' * visit VO manager [[File:Building.png|frameless|alt=VO]] section in {{PerunGUILink|rel=vo/list perunGUI}} * use '''Select VO''' [[File:Building.png|frameless|alt=VO|]]from left menu to display list of VOs which you can manage * select VO, which application you need to approve by clicking on the row with VO name * click on the button '''Applications''' at upper bar '''OR''' click to '''Applications''' [[File:application_from_storage.png|frameless|alt=VO|]] option at left menu * find the user application to approve -- to find it easily, sort a list by adjusted state ** check application state *** if the application is in the state ''Approved'' -- somebody else (another VO manager) has approved it and the whole process is done *** if the application is in the state ''Waiting for mail verification'' -- user's e-mail has not been verified yet so you have to wait until verification will proceed *** if the application is in the state ''Submitted'' you can click on the row with selected application to display its detail, after this detail is checked, approval will be done by pushing the button '''Approve''' [[File:accept.png|frameless|alt=VO|]] *** if you need to select all ''Submitted'' and ''Waiting for mail verification'' application select sate ''Pending'' for filter at the top of page. [[Category:Tutorials]] [[Category:Perun]] [[Category:English]] k4jd4u8w0wq3xxogbv75yxybz81odol 0 92 300 299 2016-01-23T15:48:49Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Use cases|Back to Use cases]] == Remove member from VO == [[My role is VO MANAGER|''VO MANAGER]] ''role needed'' * visit {{PerunGUILink|rel=vo/list perunGUI VOs pages}} * use '''Select VO''' [[File:Building.png|frameless|alt=VO|]] from left menu to display list of VOs which you can manage * click on the row with a VO name to select the VO from which you want to remove a member * click on the button '''Members''' at the upper bar '''OR''' click to '''Members''' [[File:user_green.png|frameless|alt=VO|]]option at left menu * write member's name or logname into searching box and press the '''Search''' [[File:find.png|frameless|alt=VO|]] button - rows with matching members are displayed * select the member by ticking the check-box and click on the button '''Remove''' [[File:remove.png|frameless|alt=VO|]] * a confirmation window with the name of the selected member is displayed and option [[File:accept.png|frameless|alt=VO|]] '''OK''' causes removing member from the VO * click on [[File:stop.png|frameless|alt=VO|]] '''Cancel''' if you have just changed your mind [[Category:Tutorials]] [[Category:Perun]] [[Category:English]] g0bb37u5yz6c0110qzjjqiot79trani 0 93 302 301 2016-01-23T15:48:49Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Use cases|Back to Use cases]] == Create group in VO == [[My role is VO MANAGER|''VO MANAGER]] ''role needed'' * visit {{PerunGUILink|rel=vo/list perunGUI VOs pages}} * use '''Select VO''' [[File:Building.png|frameless|alt=VO|]] from the left menu to display a list of VOs which you can manage * click on the row with a VO name to select the VO in which you want to create a new group * the simplest way to create a group is click on the [[File:add.png|frameless|alt=VO|]] '''Create group...''' button in the ''Quick tools'' area on option '''Overview''' * other possibilities are click on the button '''Group''' at upper bar '''OR''' click on the '''Group''' [[File:group.png|frameless|alt=VO|]] option at the left menu and then press button '''Create...''' [[File:add.png|frameless|alt=VO|]] * in all cases is displayed frame with two fields to fill: ** ''Name'' for a group name ** ''Description'' for a purpose of a group creation ** check-box ''As sub-group'' for creation a group as a sub-group, in that case the menu ''Parent group'' is displayed to select parent group for newly created group * click on the '''Create''' [[File:add.png|frameless|alt=VO|]] button to create the new group * click on '''Cancel''' [[File:stop.png|frameless|alt=VO|]] if you have just changed your mind [[Category:Tutorials]] [[Category:Perun]] [[Category:English]] 6flftxu8h8b1v6q3arqdazz2fkbv2wi 0 94 304 303 2016-01-23T15:48:49Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Use cases|Back to Use cases]] == Remove group from VO == [[My role is VO MANAGER|''VO MANAGER]] ''role needed'' * visit {{PerunGUILink|rel=vo/list perunGUI VOs pages}} * use '''Select VO''' [[File:Building.png|frameless|alt=VO|]] from the left menu to display a list of VOs which you can manage * click on the row with a VO name to select the VO from which you want to remove group * click on the button '''Groups''' at upper bar '''OR''' click to '''Groups''' [[File:group.png|frameless|alt=VO|]] option at the left menu * select the group to be removed by ticking check-box and press the button '''Delete''' [[File:remove.png|frameless|alt=VO|]] * confirmation window with the name of selected group is displayed and option '''OK''' [[File:accept.png|frameless|alt=VO|]] causes deleting this group from the VO * click on '''Cancel''' [[File:stop.png|frameless|alt=VO|]] if you have just changed your mind [[Category:Tutorials]] [[Category:Perun]] [[Category:English]] tljbsy1282b9qxowsugprlcfx9n8uzv 0 95 306 305 2016-01-23T15:48:49Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Use cases|Back to Use cases]] == Add member to group == Please choose one of three branches of First steps according to your role. Then follow Final part which is same for every role type. ===First steps=== First steps when your role is both [[My role is VO MANAGER|''VO MANAGER'']] and [[My role is GROUP MANAGER|''GROUP MANAGER'']]. * visit {{PerunGUILink|rel=vo GUI}} * Go to '''Group manager''' [[File:group.png|frameless|alt=group|]] in left menu and select '''Select group''' [[File:group.png|frameless|alt=group|]] * select the VO in which you want to work by popup menu '''Selected VO''' [[File:Building.png|frameless|alt=VO|]] * click on the row with an appropriate group name to select the group First steps when your role is [[My role is GROUP MANAGER|''GROUP MANAGER'']] in one VO only. * visit {{PerunGUILink|rel=vo GUI}} * click on the '''Group manager''' [[File:group.png|frameless|alt=group|]] in the left menu * click on the '''Select group''' [[File:group.png|frameless|alt=group|]] and click to row with an appropriate group name to select the group First steps when your role is [[My role is GROUP MANAGER|''GROUP MANAGER'']] in more than one VO. * visit {{PerunGUILink|rel=vo GUI}} * click on the '''Select groups''' [[File:group.png|frameless|alt=VO|]] in the '''Group manager''' left menu * select the VO in which you want to work by popup menu '''Selected VO''' * click on the row with an appropriate group name to select the group ===Final part=== Final steps same for all role types: * click on the button '''Members''' at the upper bar '''OR''' click to the '''Members''' [[File:user_green.png|frameless|alt=VO|]] option at left menu (it could be default option) * press the [[File:add.png|frameless|alt=VO|]] '''Add...''' button to find members to add * fill name or logname of new member into searching area and press the '''Search''' [[File:find.png|frameless|alt=VO|]] button * list of appropriating members is displayed * when no matching result is found, check whether user is a member of the parent VO (or ask VO manager to do it) * select new member by ticking check-box and click the [[File:add.png|frameless|alt=VO|]] '''Add''' button - the new member is added into the group * list of newly added members is dispayed under searching area * to finish adding of members click the [[File:cross.png|frameless|alt=VO|]] '''Close''' button [[Category:Tutorials]] [[Category:Perun]] [[Category:English]] 3v7tiem92y1teblq9g21vcjd21f6tk8 0 96 308 307 2016-01-23T15:48:50Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Use cases|Back to Use cases]] == Remove member from group == [[My role is GROUP MANAGER|''GROUP MANAGER'']] ''or higher role needed'' * visit {{PerunGUILink|rel=vo/list perunGUI VOs pages}} * if you are ''GROUP MANAGER'' of several groups in different VOs you have to select VO in which you want to work by popup menu '''Selected VO''' [[File:Building.png|frameless|alt=VO|]] * select group by click on the row with appropriate group name * click on the button '''Members''' at upper bar '''OR''' click to '''Members''' [[File:user_green.png|frameless|alt=VO|]] option at left menu (it could be default option) * fill the name or the logname of the member who has to be removed into searching box and press the button [[File:filter.png|frameless|alt=VO|]] '''Filter''' * select the group member by ticking check-box and click on the button [[File:remove.png|frameless|alt=VO|]] '''Remove''' * confirmation window with the name of the selected member is displayed and option [[File:accept.png|frameless|alt=VO|]] '''OK''' causes removing the member from the group * click on [[File:stop.png|frameless|alt=VO|]] '''Cancel''' if you have just changed your mind [[Category:Tutorials]] [[Category:Perun]] [[Category:English]] f9c6ki9e0agya92pfz770vmla4goiip 0 97 310 309 2016-01-23T15:48:50Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Use cases|Back to Use cases]] == Add VO manager == [[My role is VO MANAGER|''VO MANAGER]] ''role needed'' * visit perunGUI VOs [[File:Building.png|frameless|alt=VO|]] pages in {{PerunGUILink|rel=vo/list GUI}} * use '''Select VO''' [[File:Building.png|frameless|alt=VO|]] from the left menu to display a list of VOs which you can manage * click on the row with a VO name to select the VO which new manager you want to add * click on the '''Manager''' [[File:administrator.png|frameless|alt=VO|]] button at upper bar to display a list of actual VO managers * click on the [[File:add.png|frameless|alt=VO|]] '''Add...''' button to find manager to add * fill the name or the logname of new manager into searching box and press the '''Search''' [[File:find.png|frameless|alt=VO|]] button * a list of matching managers is displayed * select the new manager by ticking check-box and click the '''Add''' [[File:add.png|frameless|alt=VO|]] button then the new manager is added to the VO * list of added managers is displayed under search field * to finish adding of managers click the '''Close''' [[File:cross.png|frameless|alt=VO|]] button [[Category:Tutorials]] [[Category:Perun]] [[Category:English]] 40gy589u6z7552u9qq2x7f0gdurcrrf 0 98 312 311 2016-01-23T15:48:50Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Use cases|Back to Use cases]] == Remove VO manager == [[My role is VO MANAGER|''VO MANAGER]] ''role needed'' * visit {{PerunGUILink|rel=vo/list perunGUI VOs pages}} * use '''Select VO''' [[File:Building.png|frameless|alt=VO|]] from the left menu to display a list of VOs which you can manage * click on the row with a VO name to select the VO from which you want to remove a manager * click on the '''Manager''' [[File:administrator.png|frameless|alt=VO|]] button at the upper bar to display a list of actual VO managers * select the manager by ticking check-box and press the [[File:remove.png|frameless|alt=VO|]] '''Remove''' button * confirmation window with a name of the selected manager is displayed and option [[File:accept.png|frameless|alt=VO|]] '''OK''' causes removing the manager from the VO * click on [[File:stop.png|frameless|alt=VO|]] '''Cancel''' if you have just changed your mind [[Category:Tutorials]] [[Category:Perun]] [[Category:English]] ife9p8rewujf6hswd3mm0wubivyjwfm 0 99 314 313 2016-01-23T15:48:50Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Use cases|Back to Use cases]].....OR.....[[Less frequent tasks|Back to Less frequent tasks]] === Add member to VO without application === [[My role is VO MANAGER|''VO MANAGER'']] ''role needed'' * visit {{PerunGUILink|rel=vo/list perunGUI VOs pages}} * use '''Select VO''' [[File:Building.png|frameless|alt=VO|]] from the left menu to display list of VOs which you can manage * click on the row with VO name to select VO into which you want to add a member * click on the button '''Members''' at upper bar '''OR''' click on the '''Members''' [[File:user_green.png|frameless|alt=VO|]] option at the left menu * press the '''Add...''' [[File:add.png|frameless|alt=VO|]] button to find a member to add ** fill a name or a logname of the new member into searching box ** press [[File:find.png|frameless|alt=VO|]] '''Search''' button ** a list of matching members is displayed ** select the new member by ticking check-box and click the [[File:add.png|frameless|alt=VO|]] '''Add''' button so the new member is added into the VO *** you can use '''Invite selected''' [[File:email.png|frameless|alt=VO|]] button to send invitation mail to selected user, so the user is invited to fill application form to VO ** you can fill next user's name or logname and repeat last 3 steps ** list of added users is displayed under searching box * pressing of [[File:cross.png|frameless|alt=VO|]] '''Close''' button is adding finished * check the status of member ** fill the name of newly added member into the searching box and click the [[File:find.png|frameless|alt=VO|]] '''Search''' button ** member status is first column at displayed member's row ** if status is not ''VALID'' you can try to change it by click on the status-icon ** small frame [[File:disk.png|frameless|alt=VO|]] '''Change member's status''' is displayed and you can choose ''VALID'' status ** if '''Error frame''' occured, check and correct member's attributes or ask for support's help by the [[File:email.png|frameless|alt=VO|]] '''Report error''' button [[Category:Tutorials]] [[Category:Perun]] [[Category:English]] aeef82utd3td7tk297hwgok0k15ph3l 0 100 316 315 2016-01-23T15:48:51Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Use cases|Back to Use cases]].....OR.....[[Less frequent tasks|Back to Less frequent tasks]] {{info|Internal system structure does not allow to assign or remove single person, as usuall groups must be used. Removing person from resource, you must think about all groups where particular person is placed. Some resources could be assigned to more groups in scope of one VO; therefore to remove member from resource fully, you have to remove him from all groups that utilize the resource. Unfortunatelly, by removing member from all groups, he could lose some of his privileges. This situation could be solved by creating new group with specific privileges or adding him to the group suitable for member.}} === Remove member from resource === [[My role is VO MANAGER|''VO MANAGER]] ''role needed'' There are several groups that contain the member and they have an access to resource. List of groups that has an access to resource can be obtained. * visit {{PerunGUILink|rel=vo/list perunGUI VOs pages}} * use '''Select VO''' [[File:Building.png|frameless|alt=VO|]] from the left menu to display list of VOs which you can manage * click on the row with VO name to select VO into which you want to work * display list of resources by selecting tab '''Resources'''[[File:server_group.png|frameless|alt=VO|]] at upper bar or in left menu * select resource by clicking on the row with resource name and detail of resource is displayed * click on the tab '''Assigned groups''' at upper bar and list of groups is displayed * for every displayed group you have to do following steps: ** select group by clicking on the row with appropriating group name ** by option '''Members''' list of members is displayed OR you can find member by filling searching area by member's name or logname and press the button [[File:filter.png|frameless|alt=VO|]] '''Filter''' ** if required member is present select him by ticking the check-box and press [[File:remove.png|frameless|alt=VO|]] '''Remove''' button ** confirmation frame appears, press [[File:accept.png|frameless|alt=VO|]] '''OK''' and the member is removed from this group ** ''PLEASE NOTE:'' member loses access to all resources that are assigned to selected group (not only to this resource, but all resources). When it is not correct, it is necessary to create new group that suits member's requirements and add the member into it. That newly created group have to be assigned to all resources where member requires keep his access. [[Category:Tutorials]] [[Category:Perun]] [[Category:English]] 0hhfy1vpcpili8gqcnbllbu8mbw9wp4 0 101 318 317 2016-01-23T15:48:51Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Use cases|Back to Use cases]].....OR.....[[Less frequent tasks|Back to Less frequent tasks]] === Create [[Conceptual_scheme_and_definition_of_terms#Account_shared_among_more_users_--_Service_member|service member]] === [[My role is VO MANAGER|''VO MANAGER]] ''role needed'' * visit {{PerunGUILink|rel=vo/list perunGUI VOs pages}} * use '''Select VO''' [[File:Building.png|frameless|alt=VO|]] from left menu to display list of VOs which you can manage * click on the row with VO name to select VO which service member you want to add into * select option '''Overview''' * the simplest way how to create service member is click on the '''Create Service member''' button in ''Quick tool'' area * frame '''1. Create service identity''' with three boxes to fill is displayed: ** ''Service member's name'' - a name describing service member to identify him (something to identify, not necessary name and surname) ** if you want to have '''login''' for service member then enter: *** ''Namespace'' - choose namespace from popup menu with possible namespaces. It must be same namespace as namespace where service member account is going to be used. *** ''Login'' - logname of new service member in selected namespace ** if you want service member to use '''certificate''' then: *** choose ''Not selected'' option for ''Namespace'' *** ''Subject DN'' - DN of certificate *** ''Issuer DN'' - DN of certification authority ** press the button [[File:add.png|frameless|alt=VO|]] '''Continue''' * frame '''2. Associate real users''' is displayed: ** fill into the searching box real user's name or logname and click on the button [[File:find.png|frameless|alt=VO|]] '''Search''' ** select by ticking check-box real user and press the button [[File:add.png|frameless|alt=VO|]] '''Add''' ** press the button '''Continue''' [[File:arrow_right.png|frameless|alt=VO|]] * frame '''3. Set password for: ''logname''''' is displayed only if ''logname'' has been set: ** fill ''Password'' and ''Re-type password'' by the same value ** press the button '''Set password''' ** entering of password can be skipped by option [[File:arrow_right.png|frameless|alt=VO|]] '''Skip''' * service member with entered name, logname and password or certificate is created and you can find him in list of VO members (option '''Members''' [[File:user_green.png|frameless|alt=VO|]]) marked as ''(Service)''[[File:user_red.png|frameless|alt=VO|]] [[Category:Tutorials]] [[Category:Perun]] [[Category:English]] a5n3v0dpbfr5q2se3f66x4z895lzk6d 0 102 320 319 2016-01-23T15:48:51Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Use cases|Back to Use cases]] [[Less frequent tasks|Back to Less frequent tasks]] === Assign service to resource === {{info|First, service must be set in the facility. Full sequence of steps including cooperation of roles is in the use case [[Configure a service]]}} [[My role is VO MANAGER|''VO MANAGER]] ''role needed'' * visit {{PerunGUILink|rel=vo/list perunGUI VOs pages}} * use '''Select VO''' [[File:Building.png|frameless|alt=VO|]] from the left menu to display list of VOs which you can manage * click on the row with VO name to select VO which you want to work with * click on the button '''Resources''' at upper bar OR click on the '''Resources''' [[File:server_group.png|frameless|alt=VO|]] option at the left menu * select resource by clicking on the row with name of required resource * click on the button '''Assigned services''' and list of recently assigned services is displayed * press the button '''Add''' [[File:add.png|frameless|alt=VO|]] and list of servives for adding is offered * select required service (or services) by ticking check-box, click the button '''Add''' [[File:add.png|frameless|alt=VO|]] again * required service is assigned to this resource {{info|It is essential to set attributes of newly added services. Please see [[Set service attributes|how to set service attributes]].}} [[Category:Tutorials]] [[Category:Perun]] [[Category:English]] 8suz5d6xe767g5flb158f4m5ehods2h 0 103 322 321 2016-01-23T15:48:51Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Use cases|Back to Use cases]].....OR.....[[Less frequent tasks|Back to Less frequent tasks]] === Set VO attributes === [[My role is VO MANAGER|''VO MANAGER]] ''role needed'' * visit {{PerunGUILink|rel=vo/list perunGUI VOs pages}} * use '''Select VO''' [[File:Building.png|frameless|alt=VO|]] from the left menu to display list of VOs which you can manage * click on the row with VO name to select VO for which you want to set attributes * if option '''Settings''' is not available at left menu then click '''''show advanced>>''''' and additional options are displayed * click on '''Settings''' * recently set attributes and their values are listed * you can update values and set them by [[File:disk.png|frameless|alt=VO|]] '''Save''' button * click '''Add''' [[File:add.png|frameless|alt=VO|]] button to display next possible attributes, you can fill values and [[File:disk.png|frameless|alt=VO|]] '''Save''' them {{info| details about setting attributes of type ''membershipExpirationRules'' you can find at [[Manuál pro správce VO#Definice pravidel pro prodlužování účtů|Rules for definition of membership expiration]]}} [[Category:Tutorials]] [[Category:Perun]] [[Category:English]] lpjqokv56e8isqk6kt923ysl9dajrm1 0 104 324 323 2016-01-23T15:48:51Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Use cases|Back to Use cases]].....OR.....[[Less frequent tasks|Back to Less frequent tasks]] === Set group attributes === [[My role is GROUP MANAGER|''GROUP MANAGER'']] ''or higher role needed'' * visit {{PerunGUILink|rel=vo/list perunGUI VOs pages}} * if list of groups which you can manage is not offered you as default, you have to do: ** use '''Select VO''' [[File:Building.png|frameless|alt=VO|]] from the left menu to display list of VOs which you can manage ** click on the row with VO name to select VO ** click on the option '''Groups''' [[File:group.png|frameless|alt=VO|]] at the left menu or at the top bar and groups from VO are listed * select group by clicking on the row with name of group, if you are permited to manage this group (your role is ''VO MANAGER'' or ''GROUP MANAGER'' of this group) detail of the group is displayed * use '''Settings''' option from top bar (or left menu) and recently set attributes relating to this group are displayed * you can update their values and [[File:disk.png|frameless|alt=VO|]] '''Save''' them * next group attributes you can set by option [[File:add.png|frameless|alt=VO|]] '''Add...''' ** list of possible attributes is displayed ** you can fill values of attributes ** by [[File:disk.png|frameless|alt=VO|]] '''Save''' set new group attributes and their values * there is possibility to set attributes for selected group on required resource: ** select resource from '''Resources''' list-box and attributes for required combination of group and resource are listed ** you can update and [[File:disk.png|frameless|alt=VO|]] '''Save''' values of these attributes ** if you click on the button [[File:add.png|frameless|alt=VO|]] '''Add...''' next possible attributes for required combination of group and resource are listed ** you can fill values and [[File:disk.png|frameless|alt=VO|]] '''Save''' them * next possibility is set atributes for selected group and combination of member and resource: ** at list-box '''Members''' you can select required member and attributes for this member are listed ** if resource in list-box '''Resources''' is selected too then attributes for combination member and resource are listed ** at both cases you can update and [[File:disk.png|frameless|alt=VO|]] '''Save''' them OR [[File:add.png|frameless|alt=VO|]] '''Add...''' and save next attributes as described above [[Category:Tutorials]] [[Category:Perun]] [[Category:English]] 4n34wzqosm7xyowplkfy9o8jyaigg4j 0 105 326 325 2016-01-23T15:48:52Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Use cases|Back to Use cases]].....OR.....[[Less frequent tasks|Back to Less frequent tasks]] === Set member attributes === ''PERUN MANAGER or [[My role is SELF|SELF]] role needed'' * visit {{PerunGUILink|rel=vo/list perunGUI VOs pages}} * use '''Select VO''' [[File:Building.png|frameless|alt=VO|]] from the left menu to display list of VOs which you can manage * click on the row with VO name to select VO into which you want to add a member * click on the button '''Members''' at upper bar '''OR''' click on the '''Members''' [[File:user_green.png|frameless|alt=VO|]] option at the left menu * fill your name or logname into search area and press the button [[File:find.png|frameless|alt=VO|]] '''Search''', then appropriated members are listed * click on the row with selected member and his detail is displayed * click on button '''Settings''' at upper part of form and member attributes and their values are displayed * you can use '''Filter''' list-box ** if you choose ''Required'' as value of filter then necessary member attributes ate listed with their values ** if you select ''All filled attributes'' as value of filter then all member attributes with saved values are displayed ** using ''Resource required'' value in filter requires select resource at '''Assigned resource''' list-box and then all attributes for combination of member and resource are listed * you can fill new values for attributes where the text in value field is black ** if text in value field is gray or field is not writable that means attribute value must not be changed by you * press [[File:disk.png|frameless|alt=VO|]] '''Save''' button to set new values of attributes * if you need to set new attribute click the button [[File:add.png|frameless|alt=VO|]] '''Add...''' and all possible member or member-resource attributes are listed * you can fill and [[File:disk.png|frameless|alt=VO|]] '''Save''' only writable attribute value fields * if you use [[File:stop.png|frameless|alt=VO|]] '''Cancel''' nothing is saved [[Category:Tutorials]] [[Category:Perun]] [[Category:English]] sle58bdoxuggs8nhgrucx3y3kwfte2q 0 106 328 327 2016-01-23T15:48:52Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Use cases|Back to Use cases]] [[Less frequent tasks|Back to Less frequent tasks]] === Set service attributes === {{info| make sure that service is [[Assign_service_to_resource|already assigned]]}} [[My role is VO MANAGER|''VO MANAGER]] ''role needed'' * visit {{PerunGUILink|rel=vo/list perunGUI VOs pages}} * use '''Select VO''' [[File:Building.png|frameless|alt=VO|]] from the left menu to display list of VOs which you can manage * click on the row with VO name to select VO which you want to work with * click on the button '''Resources''' at upper bar OR click on the '''Resources''' [[File:server_group.png|frameless|alt=VO|]] option at the left menu * select resource by clicking on the row with name of required resource and detail of resource is displayed * press the button '''Service settings''' * at area '''Selected service''' are listed available services in list-box and you can choose required service ** if you tick off check-box '''Offer available only''' list-box contains all possible services and you can select from them * attributes which setting is required for selected service are displayed with its defaut values * if you choose '''Selected service'''=''All'' then all required attribute values for all assigned services are displayed together at the same page * you can fill new values for attributes where the text in value field is black, if the text is gray (or field is not writable) ''VO MANAGER'' is not permited to update this value: ** you need to be ''FACILITY MANAGER'' too to set these gray text areas (so in that case the text is black and field is writable) ** ..................OR................... ** value of attribute is automatically determined from default values for VO/namespace/... * press '''Save''' [[File:disk.png|frameless|alt=VO|]] button to set new values of attributes * if you need to set new (maybe nonrequired) attribute click the button [[File:add.png|frameless|alt=VO|]] '''Add...''' and all possible service attributes are listed * you can fill and [[File:disk.png|frameless|alt=VO|]] '''Save''' only writable attribute value fields ===== [[My role is FACILITY MANAGER|''FACILITY MANAGER'']] ''role needed'' ===== * use '''Select facility''' [[File:database_server.png|frameless|alt=VO|]] option at left menu to display list of facilities which you can manage * click on the row with required facility name to select it and detail of facility is displayed * press the button '''Service settings''' * in the field '''Service''' are listed available services in list-box and you can choose required service ** if you uncheck check-box '''Offer available only''' list-box contains all possible services and you can select from them (for example if you want to prepare attributes for several services and assigning services to resource is left to ''VO MANAGER'') ** select attributes * attributes which settings are required for selected service are displayed with their defaut values or values of attribute values are empty * if you choose '''Selected service'''=''All'' then all required attribute values for all assigned services are displayed together at the same page * you can fill new values for attributes where the text in value field is black ** if text in value field is gray or field is not writable that means attribute value is generated automatically * press [[File:Disk.png|frameless|alt=VO|]] '''Save''' button to set new values of attributes * if you need to set new (maybe nonrequired) attribute click the button [[File:add.png|frameless|alt=VO|]] '''Add...''' [[File:add.png|frameless|alt=VO|]] and all possible service attributes are listed * you can fill and [[File:Disk.png|frameless|alt=VO|]] '''Save''' only writable attribute value fields {{info|details about attributes of services you can find in [[Aktuální seznam služeb|Current list of Services]]}} [[Category:Tutorials]] [[Category:Perun]] [[Category:English]] 2tvrdl6x4yqsrmy2fkspsw29z78xdvr 0 107 330 329 2016-01-23T15:48:53Z Root 1 naimportována 1 revize wikitext text/x-wiki {{Template:Perun-en-template}} Command line tool to operate with Perun database. [[Kategorie:Perun]] === Environment settings === ==== Access methods ==== * For MetaCentrum users on MetaCentrum machines: Load module ''perun-3.0'' under the MetaCentrum environment in the machine ''nympha.zcu.cz''. Consequently, run command ''perunv3''. * For others: Download Perun CLI from http://perun.metacentrum.cz/downloads/perun-cli.tar.gz Please set in the different environments: Environmental variable '''PERUN_URL''' must contain address of perun-rpc servlet: Here is the example, how it works for CESNET eInfrastructure: * "https://perun.metacentrum.cz/krb/rpc/" (Not using GSSAPI, set environmental variable PERUN_USER with value login/pass) * "https://perun.metacentrum.cz/fed/rpc/" (requires session that already exists) * "https://perun.metacentrum.cz/cert/rpc/" (accept only certificates from IGTF accredited CA) Variable '''PERL5LIB''' must follow to directory with Perun Perl libraries (e.g. ...path to CLI sources.../lib/Perun/). Variable '''PERUN_USER''' with value login@META/pass for specific authentication method (e.g. basic auth). Variable '''PERUN_RPC_TYPE''' with value "Kerberos META". '''Packages''' Following Perl modules are required: JSON::XS, LWP::Authen::Negotiate při použití GSSAPI, Text::ASCIITable, HTTP::Request::Common, Term::ReadKey, Text::Unidecode, Date::Calc. # Debian apt-get install liblwp-authen-negotiate-perl libjson-any-perl libtext-asciitable-perl libterm-readkey-perl libwww-perl libcrypt-ssleay-perl libtext-unidecode-perl libdate-calc-perl ==== Available CLI tools by functionality ==== * ''' tools to list''' ** [[#listOfPrincipalRoles|listOfPrincipalRoles]] ** [[#listOfAllowedVosOnFacility|listOfAllowedVosOnFacility]] ** [[#listOfAssignedFacilityResources|listOfAssignedFacilityResources]] ** [[#listOfAllowedFacilityUsers|listOfAllowedFacilityUsers]] ** [[#listOfAttributesDefinition|listOfAttributesDefinition]] ** [[#listOfExtSources|listOfExtSources]] ** [[#listOfFacilities|listOfFacilities]] ** [[#listOfFacilityAttributes|listOfFacilityAttributes]] ** [[#listOfFacilityManagers|listOfFacilityManagers]] ** [[#listOfHostAttributes|listOfHostAttributes]] ** [[#listOfFacilityDestinations|listOfFacilityDestinations]] ** [[#listOfFacilitiesDestinations|listOfFacilitiesDestinations]] ** [[#listOfFacilityOwners|listOfFacilityOwners]] ** [[#listOfFacilityHosts|listOfFacilityHosts]] ** [[#listOfGroupManagers|listOfGroupManagers]] ** [[#listOfGroupMembers|listOfGroupMembers]] ** [[#listOfVoGroups|listOfVoGroups]] ** [[#listOfGroupsAssignedToResource|listOfGroupsAssignedToResource]] ** [[#listOfGroupResources|listOfGroupResources]] ** [[#listOfGroupAttributes|listOfGroupAttributes]] ** [[#listOfGroupResourceAttributes|listOfGroupResourceAttributes]] ** [[#listOfRequiredGroupAttributes|listOfRequiredGroupAttributes]] ** [[#listOfMemberGroupAttributes|listOfMemberGroupAttributes]] ** [[#listOfRequiredMemberGroupAttributes|listOfRequiredMemberGroupAttributes]] ** [[#listOfMemberAttributes|listOfMemberAttributes]] ** [[#listOfMemberResourceAttributes|listOfMemberResourceAttributes]] ** [[#listOfOwners|listOfOwners]] ** [[#listOfRequiredFacilityAttributes|listOfRequiredFacilityAttributes]] ** [[#listOfRequiredResourceAttributes|listOfRequiredResourceAttributes]] ** [[#listOfRequiredMemberAttributes|listOfRequiredMemberAttributes]] ** [[#listOfRequiredMemberResourceAttributes|listOfRequiredMemberResourceAttributes]] ** [[#listOfRequiredGroupResourceAttributes|listOfRequiredGroupResourceAttributes]] ** [[#listOfRequiredServiceAttributes|listOfRequiredServiceAttributes]] ** [[#listOfResourceAttributes|listOfResourceAttributes]] ** [[#listOfResourceMembers|listOfResourceMembers]] ** [[#listOfServices|listOfServices]] ** [[#listOfServicesAssignedToResource|listOfServicesAssignedToResource]] ** [[#listOfServicesPackages|listOfServicesPackages]] ** [[#listOfUserExtSources|listOfUserExtSources]] ** [[#listOfUsers|listOfUsers]] ** [[#listOfUserAttributes|listOfUserAttributes]] ** [[#listOfRequiredUserAttributes|listOfRequiredUserAttributes]] ** [[#listOfUserFacilityAttributes|listOfUserFacilityAttributes]] ** [[#listOfRequiredUserFacilityAttributes|listOfRequiredUserFacilityAttributes]] ** [[#listOfVoManagers|listOfVoManagers]] ** [[#listOfVoExtSources|listOfVoExtSources]] ** [[#listOfVoMembers|listOfVoMembers]] ** [[#listOfVoResources|listOfVoResources]] ** [[#listOfVos|listOfVos]] ** [[#listOfVoAttributes|listOfVoAttributes]] ** [[#listOfRequiredVoAttributes|listOfRequiredVoAttributes]] ** [[#listOfAllowedUserResources|listOfAllowedUserResources]] ** [[#listOfAssignedUserFacilities|listOfAssignedUserFacilities]] ** [[#listOfResourceUsers|listOfResourceUsers]] ** [[#listOfUsersWithoutVoAssigned|listOfUsersWithoutVoAssigned]] ** [[#listOfAuditerMessages|listOfAuditerMessages]] ** [[#listOfMembersBySearcher|listOfMembersBySearcher]] ** [[#listOfPublications|listOfPublications]] ** [[#listOfNotifReceivers|listOfNotifReceivers]] ** [[#listOfNotifRegexes|listOfNotifRegexes]] ** [[#listOfNotifTemplates|listOfNotifTemplates]] ** [[#listOfNotifTemplateMessages|listOfNotifTemplateMessages]] ** [[#listOfAllFacilitiesGroupContacts|listOfAllFacilitiesGroupContacts]] ** [[#listOfSecurityTeams|listOfSecurityTeams]] ** [[#listOfExpiredMembers|listOfExpiredMembers]] ** [[#getAttribute|getAttribute]] ** [[#getUserByMember|getUserByMember]] ** [[#getUsersByAttribute|getUsersByAttribute]] ** [[#getTaskResultsForDestinations|getTaskResultsForDestinations]] ** [[#getNotifReceiver|getNotifReceiver]] ** [[#getNotifRegex|getNotifRegex]] ** [[#getNotifTemplate|getNotifTemplate]] ** [[#getNotifTemplateMessage|getNotifTemplateMessage]] ** [[#getRelatedRegexes|getRelatedRegexes]] ** [[#getContactsOfFacility|getContactsOfFacility]] ** [[#getContactsAssignedToGroup|getContactsAssignedToGroup]] ** [[#getContactsAssignedToOwner|getContactsAssignedToOwner]] ** [[#getContactsAssignedToUser|getContactsAssignedToUser]] ** [[#getSecurityTeamBlackList|getSecurityTeamBlackList]] ** [[#getFacilityBlackList|getFacilityBlackList]] ** [[#getSecurityTeamDetails|getSecurityTeamDetails]] ** [[#getSecurityTeamManagers|getSecurityTeamManagers]] ** [[#getSecurityTeamsOfFacility|getSecurityTeamsOfFacility]] * ''' tools to create and update''' ** [[#createAttribute|createAttribute]] ** [[#createFacility|createFacility]] ** [[#createFacilityLike|createFacilityLike]] ** [[#createGroup|createGroup]] ** [[#createOwner|createOwner]] ** [[#createResource|createResource]] ** [[#createService|createService]] ** [[#createServicesPackage|createServicesPackage]] ** [[#createVo|createVo]] ** [[#createExtSource|createExtSource]] ** [[#createNotifReceiver|createNotifReceiver]] ** [[#createNotifRegex|createNotifRegex]] ** [[#createNotifTemplate|createNotifTemplate]] ** [[#createNotifTemplateMessage|createNotifTemplateMessage]] ** [[#createSecurityTeam|createSecurityTeam]] ** [[#updateUser|updateUser]] ** [[#updateGroup|updateGroup]] ** [[#updateNotifReceiver|updateNotifReceiver]] ** [[#updateNotifRegex|updateNotifRegex]] ** [[#updateNotifTemplate|updateNotifTemplate]] ** [[#updateNotifTemplateMessage|updateNotifTemplateMessage]] ** [[#updateSecurityTeam|updateSecurityTeam]] ** [[#changeVoMemberStatus|changeVoMemberStatus]] ** [[#copyForm|copyForm]] ** [[#copyRegMails|copyRegMails]] * ''' tools to add ''' ** [[#addFacilityDestination|addFacilityDestination]] ** [[#addFacilityOwner|addFacilityOwner]] ** [[#addFacilityManager|addFacilityManager]] ** [[#addHostsToFacility|addHostsToFacility]] ** [[#addFacilityContacts|addFacilityContacts]] ** [[#addGroupManager|addGroupManager]] ** [[#addGroupMember|addGroupMember]] ** [[#addRequiredAttribute|addRequiredAttribute]] ** [[#addUserExtSource|addUserExtSource]] ** [[#addVoManager|addVoManager]] ** [[#addVoExtSource|addVoExtSource]] ** [[#addVoMember|addVoMember]] ** [[#addServiceToServicesPackage|addServiceToServicesPackage]] ** [[#addDestinationsDefinedByHostsOnFacility|addDestinationsDefinedByHostsOnFacility]] ** [[#addSecurityTeamManager|addSecurityTeamManager]] ** [[#addUserToBlacklist|addUserToBlacklist]] * ''' tools to delete ''' ** [[#deleteAttribute|deleteAttribute]] ** [[#deleteFacility|deleteFacility]] ** [[#deleteGroup|deleteGroup]] ** [[#deleteHost|deleteHost]] ** [[#deleteOwner|deleteOwner]] ** [[#deleteResource|deleteResource]] ** [[#deleteService|deleteService]] ** [[#deleteServicesPackage|deleteServicesPackage]] ** [[#deleteVo|deleteVo]] ** [[#deleteVoMember|deleteVoMember]] ** [[#deleteUser|deleteUser]] ** [[#deleteSecurityTeam|deleteSecurityTeam]] * ''' tools to assign, remove and set ''' ** [[#assignGroupToResource|assignGroupToResource]] ** [[#assignServiceToResource|assignServiceToResource]] ** [[#assignRegexToTemplate|assignRegexToTemplate]] ** [[#assignSecurityTeamToFacility|assignSecurityTeamToFacility]] ** [[#removeAllFacilityAttributes|removeAllFacilityAttributes]] ** [[#removeAllFacilityDestinations|removeAllFacilityDestinations]] ** [[#removeAllGroupAttributes|removeAllGroupAttributes]] ** [[#removeAllGroupResourceAttributes|removeAllGroupResourceAttributes]] ** [[#removeAllHostAttributes|removeAllHostAttributes]] ** [[#removeAllMemberAttributes|removeAllMemberAttributes]] ** [[#removeAllMemberGroupAttributes|removeAllMemberGroupAttributes]] ** [[#removeAllMemberResourceAttributes|removeAllMemberResourceAttributes]] ** [[#removeAllResourceAttributes|removeAllResourceAttributes]] ** [[#removeAllUserAttributes|removeAllUserAttributes]] ** [[#removeAllUserFacilityAttributes|removeAllUserFacilityAttributes]] ** [[#removeAllVoAttributes|removeAllVoAttributes]] ** [[#removeFacilityAttribute|removeFacilityAttribute]] ** [[#removeFacilityDestination|removeFacilityDestination]] ** [[#removeFacilityOwner|removeFacilityOwner]] ** [[#removeFacilityManager|removeFacilityManager]] ** [[#removeFacilityContacts|removeFacilityContacts]] ** [[#removeGroupManager|removeGroupManager]] ** [[#removeGroupFromResource|removeGroupFromResource]] ** [[#removeGroupMember|removeGroupMember]] ** [[#removeGroupAttribute|removeGroupAttribute]] ** [[#removeGroupResourceAttribute|removeGroupResourceAttribute]] ** [[#removeHostAttribute|removeHostAttribute]] ** [[#removeHostsFromFacility|removeHostsFromFacility]] ** [[#removeHostsAndDestFromFacility|removeHostsAndDestFromFacility]] ** [[#removeRequiredAttribute|removeRequiredAttribute]] ** [[#removeResourceAttribute|removeResourceAttribute]] ** [[#removeMemberAttribute|removeMemberAttribute]] ** [[#removeMemberResourceAttribute|removeMemberResourceAttribute]] ** [[#removeMemberGroupAttribute|removeMemberGroupAttribute]] ** [[#removeNotifReceiver|removeNotifReceiver]] ** [[#removeNotifRegex|removeNotifRegex]] ** [[#removeNotifTemplate|removeNotifTemplate]] ** [[#removeNotifTemplateMessage|removeNotifTemplateMessage]] ** [[#removeTemplateRegexRelation|removeTemplateRegexRelation]] ** [[#removeServiceFromResource|removeServiceFromResource]] ** [[#removeServiceFromServicesPackage|removeServiceFromServicesPackage]] ** [[#removeUserExtSource|removeUserExtSource]] ** [[#removeUserAttribute|removeUserAttribute]] ** [[#removeUserFacilityAttribute|removeUserFacilityAttribute]] ** [[#removeVoManager|removeVoManager]] ** [[#removeVoExtSource|removeVoExtSource]] ** [[#removeVoAttribute|removeVoAttribute]] ** [[#removeSecurityTeamManager|removeSecurityTeamManager]] ** [[#removeSecurityTeamFromFacility|removeSecurityTeamFromFacility]] ** [[#removeUserFromBlackList|removeUserFromBlackList]] ** [[#setEntitylessAttribute|setEntitylessAttribute]] ** [[#setFacilityAttribute|setFacilityAttribute]] ** [[#setGroupAttribute|setGroupAttribute]] ** [[#setGroupResourceAttribute|setGroupResourceAttribute]] ** [[#setHostAttribute|setHostAttribute]] ** [[#setMemberAttribute|setMemberAttribute]] ** [[#setMemberGroupAttribute|setMemberGroupAttribute]] ** [[#setMemberResourceAttribute|setMemberResourceAttribute]] ** [[#setMemberStatus|setMemberStatus]] ** [[#setResourceAttribute|setResourceAttribute]] ** [[#setRole|setRole]] ** [[#unsetRole|unsetRole]] ** [[#setUserAttribute|setUserAttribute]] ** [[#setUserFacilityAttribute|setUserFacilityAttribute]] ** [[#setVoAttribute|setVoAttribute]] * '''tools to force tasks''' ** [[#forceServicePropagation|forceServicePropagation]] ** [[#forceGroupSynchronization|forceGroupSynchronization]] ** [[#doTheMagic|doTheMagic]] * '''tools to authorization and other special tools''' ** [[#makeUserPerunAdmin|makeUserPerunAdmin]] ** [[#loadExtSourcesDefinitions|loadExtSourcesDefinitions]] ** [[#listOfExpiredMembers|listOfExpiredMembers]] ==== CLI tools by entity ==== * '''tools for destinations''' ** [[#addDestinationsDefinedByHostsOnFacility|addDestinationsDefinedByHostsOnFacility]] ** [[#addFacilityDestination|addFacilityDestination]] ** [[#getTaskResultsForDestinations|getTaskResultsForDestinations]] ** [[#listOfFacilitiesDestinations|listOfFacilitiesDestinations]] ** [[#listOfFacilityDestinations|listOfFacilityDestinations]] ** [[#removeAllFacilityDestinations|removeAllFacilityDestinations]] ** [[#removeFacilityDestination|removeFacilityDestination]] * '''tools for external sources''' ** [[#addUserExtSource|addUserExtSource]] ** [[#addVoExtSource|addVoExtSource]] ** [[#createExtSource|createExtSource]] ** [[#listOfExtSources|listOfExtSources]] ** [[#listOfUserExtSources|listOfUserExtSources]] ** [[#listOfVoExtSources|listOfVoExtSources]] ** [[#loadExtSourcesDefinitions|loadExtSourcesDefinitions]] ** [[#removeUserExtSource|removeUserExtSource]] ** [[#removeVoExtSource|removeVoExtSource]] * '''tools for facilities''' ** [[#addDestinationsDefinedByHostsOnFacility|addDestinationsDefinedByHostsOnFacility]] ** [[#addFacilityDestination|addFacilityDestination]] ** [[#addFacilityManager|addFacilityManager]] ** [[#addFacilityOwner|addFacilityOwner]] ** [[#addHostsToFacility|addHostsToFacility]] ** [[#addFacilityContacts|addFacilityContacts]] ** [[#createFacility|createFacility]] ** [[#createFacilityLike|createFacilityLike]] ** [[#deleteFacility|deleteFacility]] ** [[#getContactsOfFacility|getContactsOfFacility]] ** [[#getContactsAssignedToGroup|getContactsAssignedToGroup]] ** [[#getContactsAssignedToOwner|getContactsAssignedToOwner]] ** [[#getContactsAssignedToUser|getContactsAssignedToUser]] ** [[#listOfAllowedFacilityUsers|listOfAllowedFacilityUsers]] ** [[#listOfAllowedVosOnFacility|listOfAllowedVosOnFacility]] ** [[#listOfAssignedFacilityResources|listOfAssignedFacilityResources]] ** [[#listOfAssignedUserFacilities|listOfAssignedUserFacilities]] ** [[#listOfFacilities|listOfFacilities]] ** [[#listOfFacilitiesDestinations|listOfFacilitiesDestinations]] ** [[#listOfFacilityAttributes|listOfFacilityAttributes]] ** [[#listOfFacilityDestinations|listOfFacilityDestinations]] ** [[#listOfFacilityHosts|listOfFacilityHosts]] ** [[#listOfFacilityManagers|listOfFacilityManagers]] ** [[#listOfFacilityOwners|listOfFacilityOwners]] ** [[#listOfRequiredFacilityAttributes|listOfRequiredFacilityAttributes]] ** [[#listOfRequiredUserFacilityAttributes|listOfRequiredUserFacilityAttributes]] ** [[#listOfUserFacilityAttributes|listOfUserFacilityAttributes]] ** [[#listOfAllFacilitiesGroupContacts|listOfAllFacilitiesGroupContacts]] ** [[#removeAllFacilityAttributes|removeAllFacilityAttributes]] ** [[#removeAllFacilityDestinations|removeAllFacilityDestinations]] ** [[#removeAllUserFacilityAttributes|removeAllUserFacilityAttributes]] ** [[#removeFacilityAttribute|removeFacilityAttribute]] ** [[#removeFacilityDestination|removeFacilityDestination]] ** [[#removeFacilityManager|removeFacilityManager]] ** [[#removeFacilityOwner|removeFacilityOwner]] ** [[#removeHostsAndDestFromFacility|removeHostsAndDestFromFacility]] ** [[#removeHostsFromFacility|removeHostsFromFacility]] ** [[#removeUserFacilityAttribute|removeUserFacilityAttribute]] ** [[#removeFacilityContacts|removeFacilityContacts]] ** [[#setFacilityAttribute|setFacilityAttribute]] ** [[#setUserFacilityAttribute|setUserFacilityAttribute]] * '''tools for groups''' ** [[#addGroupManager|addGroupManager]] ** [[#addGroupMember|addGroupMember]] ** [[#assignGroupToResource|assignGroupToResource]] ** [[#createGroup|createGroup]] ** [[#deleteGroup|deleteGroup]] ** [[#forceGroupSynchronization|forceGroupSynchronization]] ** [[#getContactsAssignedToGroup|getContactsAssignedToGroup]] ** [[#listOfGroupAttributes|listOfGroupAttributes]] ** [[#listOfGroupManagers|listOfGroupManagers]] ** [[#listOfGroupMembers|listOfGroupMembers]] ** [[#listOfGroupResourceAttributes|listOfGroupResourceAttributes]] ** [[#listOfGroupResources|listOfGroupResources]] ** [[#listOfGroupsAssignedToResource|listOfGroupsAssignedToResource]] ** [[#listOfMemberGroupAttributes|listOfMemberGroupAttributes]] ** [[#listOfRequiredGroupAttributes|listOfRequiredGroupAttributes]] ** [[#listOfRequiredGroupResourceAttributes|listOfRequiredGroupResourceAttributes]] ** [[#listOfRequiredMemberGroupAttributes|listOfRequiredMemberGroupAttributes]] ** [[#listOfVoGroups|listOfVoGroups]] ** [[#removeAllGroupAttributes|removeAllGroupAttributes]] ** [[#removeAllGroupResourceAttributes|removeAllGroupResourceAttributes]] ** [[#removeAllMemberGroupAttributes|removeAllMemberGroupAttributes]] ** [[#removeGroupAttribute|removeGroupAttribute]] ** [[#removeGroupFromResource|removeGroupFromResource]] ** [[#removeGroupManager|removeGroupManager]] ** [[#removeGroupMember|removeGroupMember]] ** [[#removeGroupResourceAttribute|removeGroupResourceAttribute]] ** [[#removeMemberGroupAttribute|removeMemberGroupAttribute]] ** [[#setGroupAttribute|setGroupAttribute]] ** [[#setGroupResourceAttribute|setGroupResourceAttribute]] ** [[#setMemberGroupAttribute|setMemberGroupAttribute]] ** [[#updateGroup|updateGroup]] * '''tools for members''' ** [[#addGroupMember|addGroupMember]] ** [[#addVoMember|addVoMember]] ** [[#changeVoMemberStatus|changeVoMemberStatus]] ** [[#deleteVoMember|deleteVoMember]] ** [[#getUserByMember|getUserByMember]] ** [[#listOfGroupMembers|listOfGroupMembers]] ** [[#listOfMemberAttributes|listOfMemberAttributes]] ** [[#listOfMemberGroupAttributes|listOfMemberGroupAttributes]] ** [[#listOfMemberResourceAttributes|listOfMemberResourceAttributes]] ** [[#listOfMembersBySearcher|listOfMembersBySearcher]] ** [[#listOfRequiredMemberAttributes|listOfRequiredMemberAttributes]] ** [[#listOfRequiredMemberGroupAttributes|listOfRequiredMemberGroupAttributes]] ** [[#listOfRequiredMemberResourceAttributes|listOfRequiredMemberResourceAttributes]] ** [[#listOfResourceMembers|listOfResourceMembers]] ** [[#listOfVoMembers|listOfVoMembers]] ** [[#removeAllMemberAttributes|removeAllMemberAttributes]] ** [[#removeAllMemberGroupAttributes|removeAllMemberGroupAttributes]] ** [[#removeAllMemberResourceAttributes|removeAllMemberResourceAttributes]] ** [[#removeGroupMember|removeGroupMember]] ** [[#removeMemberAttribute|removeMemberAttribute]] ** [[#removeMemberGroupAttribute|removeMemberGroupAttribute]] ** [[#removeMemberResourceAttribute|removeMemberResourceAttribute]] ** [[#setMemberAttribute|setMemberAttribute]] ** [[#setMemberGroupAttribute|setMemberGroupAttribute]] ** [[#setMemberResourceAttribute|setMemberResourceAttribute]] ** [[#setMemberStatus|setMemberStatus]] * '''tools for notifications''' ** [[#assignRegexToTemplate|assignRegexToTemplate]] ** [[#createNotifReceiver|createNotifReceiver]] ** [[#createNotifRegex|createNotifRegex]] ** [[#createNotifTemplate|createNotifTemplate]] ** [[#createNotifTemplateMessage|createNotifTemplateMessage]] ** [[#getNotifReceiver|getNotifReceiver]] ** [[#getNotifRegex|getNotifRegex]] ** [[#getNotifTemplate|getNotifTemplate]] ** [[#getNotifTemplateMessage|getNotifTemplateMessage]] ** [[#getRelatedRegexes|getRelatedRegexes]] ** [[#listOfNotifReceivers|listOfNotifReceivers]] ** [[#listOfNotifRegexes|listOfNotifRegexes]] ** [[#listOfNotifTemplates|listOfNotifTemplates]] ** [[#listOfNotifTemplateMessages|listOfNotifTemplateMessages]] ** [[#removeNotifReceiver|removeNotifReceiver]] ** [[#removeNotifRegex|removeNotifRegex]] ** [[#removeNotifTemplate|removeNotifTemplate]] ** [[#removeNotifTemplateMessage|removeNotifTemplateMessage]] ** [[#removeTemplateRegexRelation|removeTemplateRegexRelation]] ** [[#updateNotifReceiver|updateNotifReceiver]] ** [[#updateNotifRegex|updateNotifRegex]] ** [[#updateNotifTemplate|updateNotifTemplate]] ** [[#updateNotifTemplateMessage|updateNotifTemplateMessage]] ** [[#startNotifications|startNotifications]] ** [[#stopNotifications|stopNotifications]] * '''tools for owners''' ** [[#addFacilityOwner|addFacilityOwner]] ** [[#createOwner|createOwner]] ** [[#deleteOwner|deleteOwner]] ** [[#getContactsAssignedToOwner|getContactsAssignedToOwner]] ** [[#listOfFacilityOwners|listOfFacilityOwners]] ** [[#listOfOwners|listOfOwners]] ** [[#removeFacilityOwner|removeFacilityOwner]] * '''tools for resources''' ** [[#assignGroupToResource|assignGroupToResource]] ** [[#assignServiceToResource|assignServiceToResource]] ** [[#createResource|createResource]] ** [[#deleteResource|deleteResource]] ** [[#listOfAllowedUserResources|listOfAllowedUserResources]] ** [[#listOfAssignedFacilityResources|listOfAssignedFacilityResources]] ** [[#listOfGroupResourceAttributes|listOfGroupResourceAttributes]] ** [[#listOfGroupResources|listOfGroupResources]] ** [[#listOfGroupsAssignedToResource|listOfGroupsAssignedToResource]] ** [[#listOfMemberResourceAttributes|listOfMemberResourceAttributes]] ** [[#listOfRequiredGroupResourceAttributes|listOfRequiredGroupResourceAttributes]] ** [[#listOfRequiredMemberResourceAttributes|listOfRequiredMemberResourceAttributes]] ** [[#listOfRequiredResourceAttributes|listOfRequiredResourceAttributes]] ** [[#listOfResourceAttributes|listOfResourceAttributes]] ** [[#listOfResourceMembers|listOfResourceMembers]] ** [[#listOfResourceUsers|listOfResourceUsers]] ** [[#listOfServicesAssignedToResource|listOfServicesAssignedToResource]] ** [[#listOfVoResources|listOfVoResources]] ** [[#removeAllGroupResourceAttributes|removeAllGroupResourceAttributes]] ** [[#removeAllMemberResourceAttributes|removeAllMemberResourceAttributes]] ** [[#removeAllResourceAttributes|removeAllResourceAttributes]] ** [[#removeGroupFromResource|removeGroupFromResource]] ** [[#removeGroupResourceAttribute|removeGroupResourceAttribute]] ** [[#removeMemberResourceAttribute|removeMemberResourceAttribute]] ** [[#removeResourceAttribute|removeResourceAttribute]] ** [[#removeServiceFromResource|removeServiceFromResource]] ** [[#setGroupResourceAttribute|setGroupResourceAttribute]] ** [[#setMemberResourceAttribute|setMemberResourceAttribute]] ** [[#setResourceAttribute|setResourceAttribute]] * '''tools for services''' ** [[#addServiceToServicesPackage|addServiceToServicesPackage]] ** [[#assignServiceToResource|assignServiceToResource]] ** [[#createService|createService]] ** [[#createServicesPackage|createServicesPackage]] ** [[#deleteService|deleteService]] ** [[#deleteServicesPackage|deleteServicesPackage]] ** [[#forceServicePropagation|forceServicePropagation]] ** [[#listOfRequiredServiceAttributes|listOfRequiredServiceAttributes]] ** [[#listOfServices|listOfServices]] ** [[#listOfServicesAssignedToResource|listOfServicesAssignedToResource]] ** [[#listOfServicesPackages|listOfServicesPackages]] ** [[#removeServiceFromResource|removeServiceFromResource]] ** [[#removeServiceFromServicesPackage|removeServiceFromServicesPackage]] * '''tools for user''' ** [[#addUserExtSource|addUserExtSource]] ** [[#deleteUser|deleteUser]] ** [[#getUserByMember|getUserByMember]] ** [[#getUsersByAttribute|getUsersByAttribute]] ** [[#getContactsAssignedToUser|getContactsAssignedToUser]] ** [[#listOfAllowedFacilityUsers|listOfAllowedFacilityUsers]] ** [[#listOfAllowedUserResources|listOfAllowedUserResources]] ** [[#listOfAssignedUserFacilities|listOfAssignedUserFacilities]] ** [[#listOfRequiredUserAttributes|listOfRequiredUserAttributes]] ** [[#listOfRequiredUserFacilityAttributes|listOfRequiredUserFacilityAttributes]] ** [[#listOfResourceUsers|listOfResourceUsers]] ** [[#listOfUserAttributes|listOfUserAttributes]] ** [[#listOfUserExtSources|listOfUserExtSources]] ** [[#listOfUserFacilityAttributes|listOfUserFacilityAttributes]] ** [[#listOfUsers|listOfUsers]] ** [[#listOfUsersWithoutVoAssigned|listOfUsersWithoutVoAssigned]] ** [[#makeUserPerunAdmin|makeUserPerunAdmin]] ** [[#removeAllUserAttributes|removeAllUserAttributes]] ** [[#removeAllUserFacilityAttributes|removeAllUserFacilityAttributes]] ** [[#removeUserAttribute|removeUserAttribute]] ** [[#removeUserExtSource|removeUserExtSource]] ** [[#removeUserFacilityAttribute|removeUserFacilityAttribute]] ** [[#setUserAttribute|setUserAttribute]] ** [[#setUserFacilityAttribute|setUserFacilityAttribute]] ** [[#updateUser|updateUser]] * '''tools for VOs''' ** [[#addVoExtSource|addVoExtSource]] ** [[#addVoManager|addVoManager]] ** [[#addVoMember|addVoMember]] ** [[#changeVoMemberStatus|changeVoMemberStatus]] ** [[#createVo|createVo]] ** [[#deleteVo|deleteVo]] ** [[#deleteVoMember|deleteVoMember]] ** [[#listOfAllowedVosOnFacility|listOfAllowedVosOnFacility]] ** [[#listOfRequiredVoAttributes|listOfRequiredVoAttributes]] ** [[#listOfUsersWithoutVoAssigned|listOfUsersWithoutVoAssigned]] ** [[#listOfVoAttributes|listOfVoAttributes]] ** [[#listOfVoExtSources|listOfVoExtSources]] ** [[#listOfVoGroups|listOfVoGroups]] ** [[#listOfVoManagers|listOfVoManagers]] ** [[#listOfVoMembers|listOfVoMembers]] ** [[#listOfVoResources|listOfVoResources]] ** [[#listOfVos|listOfVos]] ** [[#removeAllVoAttributes|removeAllVoAttributes]] ** [[#removeVoAttribute|removeVoAttribute]] ** [[#removeVoExtSource|removeVoExtSource]] ** [[#removeVoManager|removeVoManager]] ** [[#setVoAttribute|setVoAttribute]] * '''tools for SecurityTeams''' ** [[#addSecurityTeamManager|addSecurityTeamManager]] ** [[#assignSecurityTeamToFacility|assignSecurityTeamToFacility]] ** [[#createSecurityTeam|createSecurityTeam]] ** [[#deleteSecurityTeam|deleteSecurityTeam]] ** [[#getSecurityTeamDetails|getSecurityTeamDetails]] ** [[#getSecurityTeamManagers|getSecurityTeamManagers]] ** [[#getSecurityTeamsOfFacility|getSecurityTeamsOfFacility]] ** [[#listOfSecurityTeams|listOfSecurityTeams]] ** [[#removeSecurityTeamFromFacility|removeSecurityTeamFromFacility]] ** [[#removeSecurityTeamManager|removeSecurityTeamManager]] ** [[#updateSecurityTeam|updateSecurityTeam]] ** [[#addUserToBlacklist|addUserToBlacklist]] ** [[#getFacilityBlackList|getFacilityBlackList]] ** [[#getSecurityTeamBlackList|getSecurityTeamBlackList]] ** [[#removeUserFromBlackList|removeUserFromBlackList]] -------------------------------------------------- ==== addFacilityDestination ==== Adds facility destination. Facility, service and destination and type are required fields. -------------------------------------- Available options: {| width=80% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --serviceId || '''-s''' || service id |- | --destination || '''-D''' || destination string |- | --type || '''-t''' || destination type (host/email/url) |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== addFacilityOwner ==== Adds facility owner. Facility and owner are required fields. -------------------------------------- Available options: {| width=55% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --ownerId || '''-o''' || owner id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== addFacilityManager ==== Adds a Facility manager. User Id and Facility are required fields. -------------------------------------- Available options: {| width=55% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --facilityType || '''-t''' || facility type |- | --userId || '''-u''' || user id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== addHostsToFacility ==== Adds facility hosts. Facility and hosts are required fields. -------------------------------------- Available options: {| width=55% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --hosts || '''-H''' || list Of Hosts |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== addGroupManager ==== Adds a group manager. User id and group id or group name together with vo id or vo short name are required fields. ------------------------------------ Available options: {| width=55% | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --userId || '''-u''' || user id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== addGroupMember ==== Adds a member to a group. Member id and group id or group name together with vo id or vo short name are required fields. ------------------------------------ Available options: {| width=55% | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --memberId || '''-m''' || member id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== addRequiredAttribute ==== Adds required attribute. Service and attribute are required fields. -------------------------------------- Available options: {| width=55% | --serviceId || '''-s''' || service id |- | --attributeId || '''-a''' || attribute id |- | --attributeName || '''-A''' || attribute name including namespace |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== addUserExtSource ==== Adds an external source to the user. User id and external login and external source id or name are required fields ------------------------------------ Available options: {| width=55% | --userId || '''-u''' || user id |- | --extSourceId || '''-e''' || external source id |- | --extSourceName || '''-E''' || external source name |- | --extSourceLogin || '''-l''' || external login |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== addVoManager ==== Adds a Vo manager. User id and vo id or vo short name are required fields. ------------------------------------ Available options: {| width=55% | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --userId || '''-u''' || user id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== addVoExtSource ==== Adds an external source to the Vo. Vo id or short name and external source id or name are required fields. ------------------------------------ Available options: {| width=55% | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --extSourceId || '''-e''' || external source id |- | --extSourceName || '''-E''' || external source name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== addVoMember ==== Adds a Vo member. Search string and vo id or vo short name are required fields. Searches the users in all assigned external sources using search string. ------------------------------------ Available options: {| width=90% | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --searchString || '''-p''' || string to search in assigned external sources |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== addServiceToServicesPackage ==== Adds a service to service package. Package id or name and service id or name are required fields. ----------------------------------------------------- Available options: {| width=50% | --packageId || '''-p''' || package id |- | --packageName || '''-P''' || package name |- | --serviceId || '''-s''' || service id |- | --serviceName || '''-S''' || service name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== addDestinationsDefinedByHostsOnFacility ==== Adds facility destination. Facility and service are required fields. -------------------------------------- Available options: {| width=55% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --serviceId || '''-s''' || service id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== assignGroupToResource ==== Assigns a group to a resource. Resource id and group id or group name together with vo id or vo short name are required fields. ------------------------------------ Available options: {| width=55% | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --resource || '''-r''' || resource id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== assignServiceToResource ==== Assigns service to the resource. Resource and service are required fields. -------------------------------------- Available options: {| width=55% | --serviceId || '''-s''' || service id |- | --serviceName || '''-S''' || service name |- | --resourceId || '''-r''' || resource id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== assignRegexToTemplate ==== Assigns NotifRegex to NotifTemplate. Both Id are required. --------------------------------------- Available options: {| width=55% | --NotifTemplateId || '''-t''' || id of the NotifTemplate |- | --NotifRegexId || '''-r''' || id of the NotifRegex |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== createAttribute ==== Creates an attribute. Attribute name, namespace and type are required fields. -------------------------------------- Available options: {| width=65% | --attributeName || '''-N''' || attribute friendly name |- | --attributeNameSpace || '''-n''' || namespace |- | --attributeDisplayName || '''-e''' || attribute display name |- | --attributeType || '''-t''' || attribute type (integer/string/array) |- | --attributeDsc || '''-d''' || description |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== createFacility ==== Creates a facility. Facility name is required field. -------------------------------------- Available options: {| width=55% | --facilityName || '''-F''' || facility name |- | --facilityDsc || '''-d''' || facility description |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== createFacilityLike ==== Creates a new facility like recently created facility. ------------------------------------------------------ Available options: {| width=75% | --likeFacility || '''-l''' || name of 'like' facility |- | --facilityName || '''-F''' || facility name |- | --facilityDsc || '''-d''' || facility description |- | --nodeName || '''-n''' || name of node |- | --fromNumber || '''-o''' || numbering of nodes starts from |- | --toNumber || '''-d''' || numbering of nodes ends to |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== createGroup ==== Creates a group. Group name and vo id or vo short name are required fields. -------------------------------------- Available options: {| width=55% | --groupName || '''-G''' || group name |- | --groupDsc || '''-d''' || description |- | --parentGroupId || '''-p''' || parent group id |- | --parentGroupName || '''-P''' || parent group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== createOwner ==== Creates a owner. Owner name is required fields. -------------------------------------- Available options: {| width=65% | --ownerName || '''-O''' || owner name |- | --ownerContact || '''-c''' || owner contact |- | --ownerType || '''-t''' || owner type (administrative, technical) |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== createResource ==== Creates a resource. Resource name and facility id or name are required fields. -------------------------------------- Available options: {| width=65% | --resourceName || '''-R''' || resource name |- | --resourceDsc || '''-d''' || resource description |- | --voId || '''-v''' || VO identifier |- | --voShortName || '''-V''' || VO short name |- | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== createService ==== Creates a service. Service name and owner id are required fields. -------------------------------------- Available options: {| width=60% | --serviceName || '''-S''' || service name |- | --ownerId || '''-o''' || owner id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== createServicesPackage ==== Creates a services package. ServicesPackage name is required field. -------------------------------------- Available options: {| width=50% | --packageName || '''-P''' || package name |- | --packageDescription || '''-d''' || description |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== createVo ==== Creates a vo. Vo full and short name are required fields. -------------------------------------- Available options: {| width=85% | --voFullName || '''-n''' || vo full name |- | --voShortName || '''-V''' || vo short name (max. 16 characters) |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== createExtSource ==== Creates an External source. ExtSource name and type are required fields. -------------------------------------- Available options: {| width=97% |- | --extSourceName || '''-e''' || extSrouce name |- | --extSourceType || '''-t''' || type (KERBEROS/IDP/INTERNAL/SQL/LDAP/ISMU/X509) |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== createNotifReceiver ==== Creates a NotifReceiver. Target, typeOfReceiver and templateId are required field. --------------------------------- Available options: {| width=97% | --target || '''-r''' || target |- | --typeOfReceiver || '''-t''' || type of receiver (EMAIL_USER / EMAIL_GROUP / JABBER) |- | --templateId || '''-i''' || template id (template has to exist in db) |- | --locale || '''-l''' || locale |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== createNotifRegex ==== Creates a NotifRegex. Regex is required field. -------------------------------------- Available options: {| width=73% | --regex || '''-r''' || regex |- | --note || '''-n''' || note |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== createNotifTemplate ==== Creates a NotifTemplate. Notify trigger and at least one primary property are required fields. -------------------------------------- Available options: {| width=77% | --name || '''-n''' || name |- | --primaryProperty || '''-p''' || primary property |- | || || (key and value divide by '=', |- | || || values divide by '/', more properties allowed) |- | --sender || '''-s''' || sender |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== createNotifTemplateMessage ==== Creates a NotifTemplateMessage. TemplateId, locale and subject are required fields. -------------------------------------- Available options: {| width=55% | --templateId || '''-t''' || template id |- | --locale || '''-l''' || locale |- | --message || '''-m''' || message |- | --subject || '''-s''' || subject |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== deleteAttribute ==== Deletes an attribute. Attribute id is required. -------------------------------------- Available options: {| width=77% | --attributeId || '''-a''' || attribute id |- | --force || '''-f''' || delete also all dependent data |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== deleteFacility ==== Deletes an facility. Facility id or facility name is required. -------------------------------------------------------------- Available options: {| width=55% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== deleteGroup ==== Deletes a group. Group id or group name together with vo id or vo short name are required fields. ------------------------------------ Available options: {| width=97% | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --force || '''-f''' || removes all members and deletes all subgroups |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== deleteHost ==== Deletes an host. Host id or host name is required. ------------------------------------------ Available options: {| width=55% | --hostId || '''-i''' || host id |- | --hostName || '''-H''' || host name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== deleteOwner ==== Deletes an owner. Owner id is required. -------------------------------------- Available options: {| width=75% | --ownerId || '''-o''' || owner id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== deleteResource ==== Deletes a resource. Resource id is required. -------------------------------------- Available options: {| width=65% | --resourceId || '''-r''' || resource id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== deleteService ==== Deletes a service. Service id or name is required. -------------------------------------------------- Available options: {| width=55% | --serviceId || '''-s''' || service id |- | --serviceName || '''-S''' || service name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== deleteServicesPackage ==== Deletes a service package. ServicesPackage name or id is required field. -------------------------------------- Available options: {| width=55% | --packageName || '''-P''' || package name |- | --packageId || '''-p''' || packageId |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== deleteVo ==== Deletes a Vo. Vo id or vo short name are required fields. ------------------------------------ Available options: {| width=80% | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --force || '''-f''' || delete also all dependent data |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== deleteVoMember ==== Deletes a member. Member id is required. -------------------------------------- Available options: {| width=65% | --memberId || '''-m''' || member id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== deleteUser ==== Deletes an user. User id is required. -------------------------------------- Available options: {| width=85% | --userId || '''-u''' || user Id |- | --force || '''-f''' || force delete |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfPrincipalRoles ==== Prints list of Principal Roles --------------------------------- Available options: {| width=85% | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfAllowedVosOnFacility ==== Prints list of VOs which are allowed to use Facility. Facility is required field. --------------------------------------------------------- Available options: {| width=75% | --facilityId || '''-f''' || facility identifier |- | --facilityName || '''-F''' || facility name |- | --orderById || '''-i''' || order by numeric id |- | --orderByName || '''-n''' || order by short name (default) |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfAssignedFacilityResources ==== Prints list of Resources assigned to Facility. Facility is required field. --------------------------------------------------- Available options: {| width=70% | --facilityId || '''-f''' || facility identifier |- | --facilityName || '''-F''' || facility name |- | --orderById || '''-i''' || order by numeric Id |- | --orderByName || '''-n''' || order by name (default) |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfAllowedFacilityUsers ==== Prints list of users. --------------------------------------------------- Available options: {| width=67% | --facilityId || '''-f''' || facility identifier |- | --facilityName || '''-F''' || facility name |- | --orderById || '''-i''' || order by user's Id |- | --orderByName || '''-n''' || order by user's name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfAttributesDefinition ==== Prints list of Attributes Definition ------------------------------------------ Available options: {| width=70% | --orderById || '''-i''' || order by numeric Id |- | --orderByName || '''-n''' || order by name (default) |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfExtSources ==== Prints list of ExtSources --------------------------------- Available options: {| width=78% | --orderById || '''-i''' || order by ExtSource identifier |- | --orderByName || '''-n''' || order by ExtSource name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfFacilities ==== Prints list of Facilities ------------------------------- Available options: {| width=67% | --orderById || '''-i''' || order by numeric Id |- | --orderByName || '''-n''' || order by ShortName |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfFacilityAttributes ==== Prints list of Attributes defined on Facility. Facility is required field. --------------------------------------------------- Available options: {| width=70% | --facilityId || '''-f''' || facility idetifier |- | --facilityName || '''-F''' || facility name |- | --orderById || '''-i''' || order by numeric Id |- | --orderByName || '''-n''' || order by Name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfHostAttributes ==== Prints list of Attributes defined on Host. Host is required field. --------------------------------------------------- Available options: {| width=70% | --hostId || '''-H''' || host idetifier |- | --orderById || '''-i''' || order by numeric Id |- | --orderByName || '''-n''' || order by Name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfFacilityDestinations ==== Prints list of Destinations used for Facility and Service. Facility and service are required fields. --------------------------------------------------------------- Available options: {| width=63% | --facilityId || '''-f''' || facility identifier |- | --facilityName || '''-F''' || facility name |- | --serviceId || '''-s''' || service identifier |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfFacilitiesDestinations ==== Prints list of Destinations used for Facilities which have resource in defined VO. --------------------------------------------------------------- Available options: {| width=85% | --voId || '''-v''' || filter facilities by resources available in VO |- | || || (defined by ID) |- | --voShortName || '''-V''' || filter facilities by resources available in VO |- | || || (defined by shortName) |- | --orderById || '''-i''' || order by destination id |- | --orderByDestination || '''-n''' || order by destination name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfFacilityOwners ==== Prints list of facility Owners. Facility is required field. ----------------------------------------------------------- Available options: {| width=63% | --facilityId || '''-f''' || facility idetifier |- | --facilityName || '''-F''' || facility name |- | --orderById || '''-i''' || order by numeric ID |- | --orderByName || '''-n''' || order by name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfFacilityManagers ==== Prints list of facility Managers. Facility is required field. ----------------------------------------------------------- Available options: {| width=63% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --orderById || '''-i''' || order by Manager's identifier |- | --orderByName || '''-n''' || order by Manager's name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfFacilityHosts ==== Prints list of facility hosts. Facility is required field. ----------------------------------------------------------- Available options: {| width=63% | --facilityId || '''-f''' || facility idetifier |- | --facilityName || '''-F''' || facility name |- | --orderById || '''-i''' || order by numeric ID |- | --orderByName || '''-n''' || order by name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfGroupManagers ==== Prints list of group Managers. Group is required field. Group can by specified by it's ID or by both it's name and VO. --------------------------------- Available options: {| width=70% | --groupId || '''-g''' || Group identifier |- | --groupName || '''-G''' || Group name |- | --voId || '''-v''' || VO idetifier |- | --voShortName || '''-V''' || VO short name |- | --orderById || '''-i''' || order by Manager's identifier |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfGroupMembers ==== Prints list of group members. Group is required field. Group can by specified by it's ID or by both it's name and VO. --------------------------------- Available options: {| width=70% | --groupId || '''-g''' || Group identifier |- | --groupName || '''-G''' || Group name |- | --voId || '''-v''' || VO idetifier |- | --voShortName || '''-V''' || VO short name |- | --orderById || '''-i''' || order by member's identifier |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfVoGroups ==== Prints list of Groups in VO. VO is required filed. --------------------------- Available options: {| width=75% | --voId || '''-v''' || VO idetifier |- | --voShortName || '''-V''' || VO shortName |- | --orderById || '''-i''' || order by numeric ID (default) |- | --orderByName || '''-n''' || order by name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfGroupsAssignedToResource ==== Prints list of Groups assigned to Resource. Resource is required field. ------------------------------------------------ Available options: {| width=75% | --resourceId || '''-r''' || resource idetifier |- | --orderById || '''-i''' || order by numeric ID (default) |- | --orderByName || '''-n''' || order by name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfGroupResources ==== Tool prints list of Resources where the Group is assigned to. Group is required fields. ------------------------------ Available options: {| width=75% | --groupId || '''-g''' || group idetifier |- | --orderById || '''-i''' || order by numeric ID (default) |- | --orderByName || '''-n''' || order by name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfGroupAttributes ==== Prints list of Attributes defined on Group. Group id or group name and VO are required fields. --------------------------------------------------- Available options: {| width=75% | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | -voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --orderById || '''-i''' || order by numeric ID (default) |- | --orderByName || '''-n''' || order by name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfRequiredGroupAttributes ==== Lists all required group attributes. Group id or group name and VO are required fields. Id of resource to get services from is required. ----------------------------------------------------------------------- Available options: {| width=82% | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | -voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --resourceId || '''-r''' || id of resource to get services from |- | --orderById || '''-i''' || order by numeric ID (default) |- | --orderByName || '''-n''' || order by name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfMemberGroupAttributes ==== Prints list of Attributes for Group and Member. Group, VO and member fields are required. ----------------------------------------------------------------------- Available options: {| width=62% | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --memberId || '''-m''' || Members identifier |- | --orderById || '''-i''' || order by numeric ID |- | --orderByName || '''-n''' || order by name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfRequiredMemberGroupAttributes ==== Lists all required member group attributes. Group, VO and member fields are required. Resource id is optional. Available options: {| width=62% | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --resourceId || '''-r''' || resource id |- | --memberId || '''-m''' || member id |- | --orderById || '''-i''' || order by numeric ID |- | --orderByName || '''-n''' || order by name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfGroupResourceAttributes ==== Prints list of Attributes defined for Resource-Group. Group id or group name and VO and resource Id are required fields. --------------------------------------------------- Available options: {| width=60% | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --resourceId || '''-r''' || resource id |- | --orderById || '''-i''' || order by numeric Id |- | --orderByName || '''-n''' || order by Name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfMemberAttributes ==== Prints list of Attributes defined for Member. Member is required field. --------------------------------------------------- Available options: {| width=60% | --memberId || '''-m''' || Members identifier |- | --orderById || '''-i''' || order by numeric Id |- | --orderByName || '''-n''' || order by Name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfMemberResourceAttributes ==== Print list of Attribute for Resource and Member. Resource and member fields are required. ------------------------------------------ Available options: {| width=60% | --resourceId || '''-r''' || Resource identifier |- | --memberId || '''-m''' || Members identifier |- | --orderById || '''-i''' || order by numeric ID |- | --orderByName || '''-n''' || order by name |} ==== listOfOwners ==== Prints list of Owners --------------------------- Available options: {| width=60% | --orderById || '''-i''' || order by numeric ID |- | --orderByName || '''-n''' || order by name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfRequiredFacilityAttributes ==== Lists all required facility attributes. Facility id or name is required option. ----------------------------------------------------------------------- Available options: {| width=75% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --orderById || '''-i''' || order by attribute id |- | --orderByName || '''-n''' || order by attribute friendly name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || help |} ==== listOfRequiredResourceAttributes ==== Lists all required resource attributes. Resource id is required option. ----------------------------------------------------------------------- Available options: {| width=105% | --resourceId || '''-r''' || resource id |- | --serviceId || '''-s''' || service id - return only atributes required by this service |- | --orderById || '''-i''' || order by attribute id |- | --orderByName || '''-n''' || order by attribute friendly name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || help |} ==== listOfRequiredGroupResourceAttributes ==== Lists all required resource-group attributes. Group id or group name and VO and resource Id are required fields. ----------------------------------------------------------------------- Available options: {| width=75% | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --resourceId || '''-r''' || resource id |- | --orderById || '''-i''' || order by attribute id |- | --orderByName || '''-n''' || order by attribute friendly name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || help |} ==== listOfRequiredMemberAttributes ==== Lists all required member attributes. Member id is required option. ----------------------------------------------------------------------- Available options: {| width=75% | --memberId || '''-m''' || member id |- | --orderById || '''-i''' || order by attribute id |- | --orderByName || '''-n''' || order by attribute friendly name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || help |} ==== listOfRequiredMemberResourceAttributes ==== Lists all required member resource attributes. Resource id and member id are required options. ----------------------------------------------------------------------- Available options: {| width=95% | --resourceId || '''-r''' || resource id |- | --memberId || '''-m''' || member id |- | --optionUseRes || '''-o''' || option for using resource to get services from |- | --orderById || '''-i''' || order by attribute id |- | --orderByName || '''-n''' || order by attribute friendly name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || help |} ==== listOfRequiredServiceAttributes ==== Prints list of Attributes required for Service. Service is required field. ---------------------------------------------------- Available options: {| width=60% | --serviceId || '''-s''' || Service idetifier |- | --serviceName || '''-S''' || Service name |- | --orderById || '''-i''' || order by numeric ID |- | --orderByName || '''-n''' || order by name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfResourceAttributes ==== Prints list of Attributes defined for Resource. Resource is required field. ---------------------------------------------------- Available options: {| width=60% | --resourceId || '''-r''' || Resource idetifier |- | --orderById || '''-i''' || order by numeric ID |- | --orderByName || '''-n''' || order by name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfResourceMembers ==== Prints list of Members allowed on Resource. Resource is required field. ---------------------------------------------------- Available options: {| width=65% | --resourceId || '''-r''' || Resource idetifier |- | --orderById || '''-i''' || order by numeric ID |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfServices ==== Prints list of Services ------------------------------ Available options: {| width=60% | --orderById || '''-i''' || order by numeric ID |- | --orderByName || '''-n''' || order by name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfServicesAssignedToResource ==== Prints list of Services assigned to Resource. Resource is required field. ------------------------------------------------ Available options: {| width=60% | --resourceId || '''-r''' || Resource idetifier |- | --orderById || '''-i''' || order by numeric ID |- | --orderByName || '''-n''' || order by name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfServicesPackages ==== Prints list of Service Packages ------------------------------- Available options: {| width=60% | --orderById || '''-i''' || order by numeric ID |- | --orderByName || '''-n''' || order by name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfUserExtSources ==== List all user external sources. User id is required. ------------------------------------------------- Available options: {| width=98% | --userId || '''-u''' || user id |- | --orderById || '''-i''' || order by user external source id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || help |} ==== listOfUsers ==== Prints list of users. ------------------------------ Available options: {| width=70% | --orderById || '''-i''' || order by user's identifier |- | --orderByName || '''-n''' || order by user's name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfUserAttributes ==== Prints list of Attributes defined for User. User is required field. --------------------------------------------------- Available options: {| width=65% | --userId || '''-u''' || user's identifier |- | --orderById || '''-i''' || order by numeric Id |- | --orderByName || '''-n''' || order by Name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfRequiredUserAttributes ==== Lists all required user attributes. User id is required option. Resource id is optional. ----------------------------------------------------------------------- Available options: {| width=80% | --userId || '''-u''' || user's identifier |- | --resourceId || '''-r''' || resource to get services from |- | --orderById || '''-i''' || order by attribute Id |- | --orderByName || '''-n''' || order by friendly name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfUserFacilityAttributes ==== Lists all user_facility attributes. User id and facility id or name are required options. ----------------------------------------------------------------------- Available options: {| width=70% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --userId || '''-u''' || user's identifier |- | --orderById || '''-i''' || order by attribute Id |- | --orderByName || '''-n''' || order by friendly name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfRequiredUserFacilityAttributes ==== Lists all required user_facility attributes. User id and facility id or name are required options. Resource id is optional. ----------------------------------------------------------------------- Available options: {| width=80% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --userId || '''-u''' || user's identifier |- | --resourceId || '''-r''' || resource to get services from |- | --orderById || '''-i''' || order by attribute Id |- | --orderByName || '''-n''' || order by friendly name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfVoManagers ==== Tool prints list of VO Managers. Vo is required fields. ------------------------------ Available options: {| width=70% | --voId || '''-v''' || VO idetifier |- | --voShortName || '''-V''' || VO short name |- | --orderById || '''-i''' || order by Manager's identifier |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfVoExtSources ==== Tool prints list of VO ExtSources. Vo is required fields. --------------------------------- Available options: {| width=70% | --voId || '''-v''' || VO idetifier |- | --voShortName || '''-V''' || VO short name |- | --orderById || '''-i''' || order by ExtSource identifier |- | --orderByName || '''-n''' || order by ExtSource name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfVoMembers ==== Prints list of VO Members. Vo is required field. ------------------------------ Available options: {| width=75% | --voId || '''-v''' || VO idetifier |- | --voShortName || '''-V''' || VO short name |- | --orderById || '''-i''' || order by Member's identifier |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfVoResources ==== Prints list of VO Resources. VO is required field. --------------------------------- Available options: {| width=75% | --voId || '''-v''' || VO idetifier |- | --voShortName || '''-V''' || VO short name |- | --orderById || '''-i''' || order by Resource identifier |- | --orderByName || '''-n''' || order by Resource name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfVos ==== Prints list of all VOs --------------------------------------- Available options: {| width=65% | --orderByName || '''-n''' || order by short name |- | --orderById || '''-i''' || order by id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfVoAttributes ==== Prints list of Attributes defined on VO. VO is required field. --------------------------------------------------- Available options: {| width=70% | --voId || '''-v''' || VO identifier |- | --voShortName || '''-V''' || VO short name |- | --orderById || '''-i''' || order by numeric Id |- | --orderByName || '''-n''' || order by Name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfRequiredVoAttributes ==== Lists all required VO attributes. VO id or VO short name is required option. ----------------------------------------------------------------------- Available options: {| width=70% | --voId || '''-v''' || VO identifier |- | --voShortName || '''-V''' || VO short name |- | --orderById || '''-i''' || order by numeric Id |- | --orderByName || '''-n''' || order by Name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfAllowedUserResources ==== Prints list of Resources assigned to User. Can be limited to resources from specified facility. User is required field. --------------------------------------------------------- Available options: {| width=70% | --userId || '''-u''' || user identifier |- | --facilityId || '''-f''' || facility identifier |- | --facilityName || '''-F''' || facility name |- | --orderById || '''-i''' || order by numeric Id |- | --orderByName || '''-n''' || order by Name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfAssignedUserFacilities ==== Prints list of Facilities assigned to User. User is required field. --------------------------------------------------- Available options: {| width=77% | --userId || '''-u''' || user identifier |- | --orderById || '''-i''' || order by numeric Id |- | --orderByName || '''-n''' || order by name (default) |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfResourceUsers ==== Prints list of Users allowed on Resource. Resource is required field. ---------------------------------------------------- Available options: {| width=70% | --resourceId || '''-r''' || Resource idetifier |- | --orderById || '''-i''' || order by numeric Id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfUsersWithoutVoAssigned ==== Prints list of users. ------------------------------ Available options: {| width=70% |- | --orderById || '''-i''' || order by numeric Id |- | --orderByName || '''-n''' || order by Name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfNotifReceivers ==== Prints list of all Notif Receivers. ------------------------------ Available options: {| width=70% | --orderByTarget || '''-t''' || order by target |- | --orderById || '''-i''' || order by id (default) |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfNotifRegexes ==== Prints list of all Notif Regexes. ------------------------------ Available options: {| width=72% | --orderByNote || '''-n''' || order by note |- | --orderById || '''-i''' || order by id (default) |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfNotifTemplates ==== Prints list of all Notif Templates. ------------------------------ Available options: {| width=72% | --orderByName || '''-n''' || order by name |- | --orderById || '''-i''' || order by id (default) |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfNotifTemplateMessages ==== Prints list of all Notif TemplateMessages. ------------------------------ Available options: {| width=62% | --orderByTemplateId || '''-t''' || order by templateId |- | --orderById || '''-i''' || order by id (default) |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== getAttribute ==== Prints selected attribute for an entity. --------------------------------------------------- Available options: {| width=70% | --userId || '''-u''' || user identifier |- | --facilityId || '''-f''' || facility identifier |- | --memberId || '''-m''' || member identifier |- | --resourceId || '''-r''' || reource identifier |- | --hostId || '''-h''' || host identifier |- | --groupId || '''-g''' || group identifier |- | --voId || '''-v''' || vo identifier |- | --attributeId || '''-a''' || attribute identifier |- | --orderById || '''-i''' || order by numeric Id |- | --orderByName || '''-n''' || order by Name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== getUserByMember ==== Find user by member id. ------------------------ Available options: {| width=70% | --memberId || '''-m''' || member Id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== getUsersByAttribute ==== Find users by attribute. ------------------------ Available options: {| width=67% | --attributeName || '''-a''' || attribute name |- | --attributeValue || '''-w''' || attribute value |- | --orderByName || '''-n''' || order by user's name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== getNotifReceiver ==== Find NotifReceiver by id. Id is required field. ------------------------ Available options: {| width=57% | --NotifReceiverId || '''-i''' || NotifReceiver id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== getNotifRegex ==== Find NotifRegex by id. Id is required field. ------------------------ Available options: {| width=60% | --NotifRegexId || '''-i''' || NotifRegex id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== getNotifTemplate ==== Find NotifTemplate by id. Id is required field. ------------------------ Available options: {| width=57% | --NotifTemplateId || '''-i''' || NotifTemplate id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== getNotifTemplateMessage ==== Find NotifTemplateMessage by id. Id is required field. ------------------------ Available options: {| width=57% | --NotifTemplateMessageId || '''-i''' || NotifTemplateMessage id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== getRelatedRegexes ==== Prints list of all related NotifRegexes to NotifTemplate. NotifTemplate id is required field. --------------------------------------- Available options: {| width=62% | --NotifTemplateId || '''-i''' || id of NotifTemplate |- | --orderByNote || '''-n''' || order by note |- | --orderById || '''-d''' || order by id (default) |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeAllFacilityAttributes ==== Removes all attributes from the facility. Facility id or name is required. ---------------------------------------------------------------------------- Available options: {| width=60% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeAllHostAttributes ==== Removes all attributes from the host. Host id is required. ------------------------------------------------- Available options: {| width=87% | --hostId || '''-H''' || host id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeAllFacilityDestinations ==== Removes all facility destinations. Facility id or facility name and service id are required. ---------------------------------------------------------------------------- Available options: {| width=60% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --serviceId || '''-s''' || service id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeAllResourceAttributes ==== Removes all attributes from the resource. Resource id is required. -------------------------------------- Available options: {| width=65% | --resourceId || '''-r''' || resource id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeAllGroupResourceAttributes ==== Removes all attributes from the resource-group. Group id or group name and VO and resource id are required fields. ------------------------------------------------- Available options: {| width=55% | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --resourceId || '''-r''' || resource id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeAllMemberResourceAttributes ==== Removes all resource-member attributes. Resource id and member id are required. -------------------------------------- Available options: {| width=65% | --resourceId || '''-r''' || resource id |- | --memberId || '''-m''' || member id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeFacilityAttribute ==== Removes selected attribute from the facility. Facility id or name and attribute id are required. -------------------------------------- Available options: {| width=60% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --attributeId || '''-a''' || attribute id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeHostAttribute ==== Removes selected attribute from the host. Host id and attribute id are required. -------------------------------------- Available options: {| width=67% | --hostId || '''-H''' || host id |- | --attributeId || '''-a''' || attribute id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeFacilityDestination ==== Removes selected destination from the facility. Facility id or name, attribute id and destination are required. -------------------------------------- Available options: {| width=60% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --serviceId || '''-s''' || service id |- | --destination || '''-D''' || destination |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeFacilityOwner ==== Removes selected owner from the facility. Facility id or name and owner id are required. -------------------------------------- Available options: {| width=55% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --ownerId || '''-o''' || owner id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeFacilityManager ==== Removes selected manager from the facility. Facility id or name and manager's id are required fields. -------------------------------------- Available options: {| width=55% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --userId || '''-u''' || user id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeHostsFromFacility ==== Removes selected hosts from the facility. Facility id and one or more host id are required fields. -------------------------------------- Available options: {| width=55% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --hosts || '''-H''' || list Of Hosts |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeHostsAndDestFromFacility ==== Removes selected hosts and destinations from the facility. Facility id (or name) and one or more host name are required fields. ------------------------------------ Available options: {| width=55% | --hosts || '''-H''' || list Of Host names |- | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeGroupManager ==== Removes group manager. User id and group id or group name and Vo id or vo short name are required fields. ------------------------------------ Available options: {| width=55% | --userId || '''-u''' || user id |- | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeGroupFromResource ==== Removes a group from the resource. Resource id and group id or group name together with vo id or vo short name are required fields. ------------------------------------ Available options: {| width=55% | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --resourceId || '''-r''' || resource id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeGroupMember ==== Removes group member. Member id and group id or group name and Vo id or vo short name are required fields. ------------------------------------ Available options: {| width=55% | --memberId || '''-m''' || member id |- | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeGroupAttribute ==== Removes selected attribute from the group. Group id or group name and VO and attribute id are required fields. -------------------------------------- Available options: {| width=85% | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --attributeId || '''-a''' || attribute id |- | --attributeName || '''-A''' || attribute name including namespace |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeAllGroupAttributes ==== Removes all attributes from the group. Group id or group name and VO are required fields. ------------------------------------------------- Available options: {| width=55% | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeRequiredAttribute ==== Removes required attribute from the service. Service id and attribute id are required. -------------------------------------- Available options: {| width=60% | --serviceId || '''-s''' || service id |- | --attributeId || '''-a''' || attribute id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeResourceAttribute ==== Removes selected attribute from the resource. Resource id and attribute id are required. -------------------------------------- Available options: {| width=60% | --resourceId || '''-r''' || resource id |- | --attributeId || '''-a''' || attribute id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeMemberResourceAttribute ==== Removes selected resource-member attribute. Resource id, member id and attribute id are required. -------------------------------------- Available options: {| width=60% | --resourceId || '''-r''' || resource id |- | --memberId || '''-m''' || member id |- | --attributeId || '''-a''' || attribute id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeMemberGroupAttribute ==== Removes selected group-member attribute. Group, VO, member, and attribute fields are required. -------------------------------------- Available options: {| width=60% | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --memberId || '''-m''' || member id |- | --attributeId || '''-a''' || attribute id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeGroupResourceAttribute ==== Removes selected attribute from the resource-group. Group id or group name and VO, resource id and attribute id are required fields. -------------------------------------- Available options: {| width=90% | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --resourceId || '''-r''' || resource id |- | --attributeId || '''-a''' || attribute id |- | --attributeName || '''-A''' || attribute name including namespace |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeServiceFromResource ==== Removes service from the resource. Resource id and service id or name are required. ---------------------------------------------------- Available options: {| width=60% | --resourceId || '''-r''' || resource id |- | --serviceId || '''-s''' || service id |- | --serviceName || '''-S''' || service name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeServiceFromServicesPackage ==== Removes a service from service package. Package id or name and service id or name are required fields. ----------------------------------------------------- Available options: {| width=60% | --packageId || '''-p''' || package id |- | --packageName || '''-P''' || package name |- | --serviceId || '''-s''' || service id |- | --serviceName || '''-S''' || service name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeUserExtSource ==== Removes an external source from the user. User id and external login and external source id or name are required fields ------------------------------------ Available options: {| width=60% | --userId || '''-u''' || user id |- | --extSourceId || '''-e''' || external source id |- | --extSourceName || '''-E''' || external source name |- | --extSourceLogin || '''-l''' || external login |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeUserAttribute ==== Removes selected attribute from the user. User id and attribute id are required. -------------------------------------- Available options: {| width=62% | --userId || '''-u''' || user id |- | --attributeId || '''-a''' || attribute id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeAllUserAttributes ==== Removes all attributes from the user. User id is required. -------------------------------------- Available options: {| width=80% | --userId || '''-u''' || user id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeUserFacilityAttribute ==== Removes selected attribute from the user and facility. Facility id or name, user id and attribute id are required. -------------------------------------- Available options: {| width=55% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --userId || '''-u''' || user id |- | --attributeId || '''-a''' || attribute id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeAllUserFacilityAttributes ==== Removes all attributes from the user and facility. Facility id or name and user id are required. -------------------------------------------------- Available options: {| width=55% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --userId || '''-u''' || user id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeVoManager ==== Removes VO Manager. User id and VO id or VO short name are required fields. ------------------------------------ Available options: {| width=55% | --userId || '''-u''' || user id |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeVoExtSource ==== Removes an external source from the VO. External source id or name and VO id or VO short name are required fields. ------------------------------------ Available options: {| width=60% | --extSourceId || '''-e''' || external source id |- | --extSourceName || '''-E''' || external source name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeVoAttribute ==== Removes selected attribute from the VO. VO id or VO short name and attribute id are required. ----------------------------------------------- Available options: {| width=55% | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --attributeId || '''-a''' || attribute id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeAllVoAttributes ==== Removes all attributes from the VO. VO id or short name is required ----------------------------------------------- Available options: {| width=55% | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeMemberAttribute ==== Removes selected attribute from the member. Member id and attribute id are required. -------------------------------------- Available options: {| width=55% | --memberId || '''-m''' || member id |- | --attributeId || '''-a''' || attribute id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeAllMemberAttributes ==== Removes all attributes from the member. Member id is required. -------------------------------------- Available options: {| width=55% | --memberId || '''-m''' || member id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeAllMemberGroupAttributes ==== Removes all group-member attributes. Group, VO and member fields are required. -------------------------------------- Available options: {| width=55% | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --memberId || '''-m''' || member id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeNotifReceiver ==== Removes NotifReceiver from db by id. Id is required field. ------------------------------------ Available options: {| width=60% | --NotifReceiverId || '''-i''' || id of the NotifReceiver |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeNotifRegex ==== Removes NotifRegex from db by id. Id is required field. ------------------------ Available options: {| width=53% | --NotifRegexId || '''-i''' || NotifRegex id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeNotifTemplate ==== Removes NotifTemplate from db by id. Id is required field. ------------------------------------ Available options: {| width=60% | --NotifTemplateId || '''-i''' || id of the NotifTemplate |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeNotifTemplateMessage ==== Removes NotifTemplateMessage from db by id. Id is required field. ------------------------------------ Available options: {| width=60% | --NotifTemplateMessageId || '''-i''' || id of the NotifTemplateMessage |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeTemplateRegexRelation ==== Removes relation between NotifTemplate and NotifRegex by id. Both Id are required. ------------------------------------ Available options: {| width=63% | --NotifTemplateId || '''-t''' || id of the NotifTemplate |- | --NotifRegexId || '''-r''' || id of the NotifRegex |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== setEntitylessAttribute ==== Sets the entityless attribute. Key and attribute value and attribute id or name are required. -------------------------------------- Available options: {| width=85% | --key || '''-k''' || key |- | --attributeId || '''-a''' || attribute id |- | --attributeName || '''-A''' || attribute name including namespace |- | --attributeValue || '''-w''' || attribute value |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== setFacilityAttribute ==== Sets the facility attribute. Facility id or name and attribute value and attribute id or name are required. --------------------------------------------------------------- Available options: {| width=85% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --attributeId || '''-a''' || attribute id |- | --attributeName || '''-A''' || attribute name including namespace |- | --attributeValue || '''-w''' || attribute value |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== setHostAttribute ==== Sets the host attribute. Host id and attribute value and attribute id or name are required. --------------------------------------------------------------- Available options: {| width=85% | --hostId || '''-H''' || host id |- | --attributeId || '''-a''' || attribute id |- | --attributeName || '''-A''' || attribute name including namespace |- | --attributeValue || '''-w''' || attribute value |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== setGroupAttribute ==== Sets the group attribute. Group id or group name and VO and attribute id or name and attribute value are required fields. --------------------------------------------------------------- Available options: {| width=85% | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --attributeId || '''-a''' || attribute id |- | --attributeName || '''-A''' || attribute name including namespace |- | --attributeValue || '''-w''' || attribute value |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== setMemberAttribute ==== Sets the member attribute. Member id and attribute value and attribute id or name are required. -------------------------------------- Available options: {| width=85% | --memberId || '''-m''' || member id |- | --attributeId || '''-a''' || attribute id |- | --attributeName || '''-A''' || attribute name including namespace |- | --attributeValue || '''-w''' || attribute value |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== setMemberResourceAttribute ==== Sets the member resource attribute. Resource id and member id and attribute value and attribute id or name are required. -------------------------------------- Available options: {| width=85% | --resourceId || '''-r''' || facility id |- | --memberId || '''-m''' || member id |- | --attributeId || '''-a''' || attribute id |- | --attributeName || '''-A''' || attribute name including namespace |- | --attributeValue || '''-w''' || attribute value |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== setMemberGroupAttribute ==== Sets the member group attribute. Group, VO, attribute value and attribute id fields are required. -------------------------------------- Available options: {| width=85% | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --memberId || '''-m''' || member id |- | --attributeId || '''-a''' || attribute id |- | --attributeName || '''-A''' || attribute name including namespace |- | --attributeValue || '''-w''' || attribute value |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== setResourceAttribute ==== Sets the resource attribute. Resource id and attribute value and attribute id or name are required. -------------------------------------- Available options: {| width=90% | --resourceId || '''-r''' || resource id |- | --attributeId || '''-a''' || attribute id |- | --attributeName || '''-A''' || attribute name including namespace |- | --attributeValue || '''-w''' || attribute value |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== setGroupResourceAttribute ==== Sets the resource-group attribute. Group id or group name and VO, resource id and attribute id or name and attribute value are required fields. --------------------------------------------------------------- Available options: {| width=90% | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --resourceId || '''-r''' || resource id |- | --attributeId || '''-a''' || attribute id |- | --attributeName || '''-A''' || attribute name including namespace |- | --attributeValue || '''-w''' || attribute value |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== setUserAttribute ==== Sets the user attribute. User id and attribute value and attribute id or name are required. -------------------------------------- Available options: {| width=90% | --userId || '''-u''' || user id |- | --attributeId || '''-a''' || attribute id |- | --attributeName || '''-A''' || attribute name including namespace |- | --attributeValue || '''-w''' || attribute value |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== setUserFacilityAttribute ==== Sets the user facility attribute. Facility id or name, user id and attribute value and attribute id or name are required. --------------------------------------------------------------- Available options: {| width=90% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --userId || '''-u''' || user id |- | --attributeId || '''-a''' || attribute id |- | --attributeName || '''-A''' || attribute name including namespace |- | --attributeValue || '''-w''' || attribute value |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== setVoAttribute ==== Sets the VO attribute. VO id or VO short name and attribute value and attribute id or name are required. --------------------------------------------------------------- Available options: {| width=90% | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --attributeId || '''-a''' || attribute id |- | --attributeName || '''-A''' || attribute name including namespace |- | --attributeValue || '''-w''' || attribute value |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== setMemberStatus ==== Sets the member's status. Member id and status are required. -------------------------------------- Available options: {| width=100% | --memberId || '''-m''' || member id |- | --status || '''-s''' || VALID/INVALID/SUSPENDED/EXPIRED/DISABLED |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== setRole ==== Adds Role to user or authorizedGroup. User or group is required. Role is required. Some complementaryObject is required if not PERUNADMIN. For now only 1 object is supported to be set at once. ------------------------------------ Available options: {| width=70% | --userId || '''-u''' || user id |- | --authorizedGroupId || '''-a''' || authorizedGroup id |- | --role || '''-R''' || role name |- | --facilityId || '''-f''' || facility comp object id |- | --groupId || '''-g''' || group comp object id |- | --voId || '''-v''' || vo comp object id |- | --resourceId || '''-r''' || resource comp object id |- | --serviceId || '''-s''' || service comp object id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== unsetRole ==== Remove Role from user or authorizedGroup. User or group is required. Role is required. Some complementaryObject is required if not PERUNADMIN. For now only 1 object is supported to be set at once. ------------------------------------ Available options: {| width=70% | --userId || '''-u''' || user id |- | --authorizedGroupId || '''-a''' || authorizedGroup id |- | --role || '''-R''' || role name |- | --facilityId || '''-f''' || facility comp object id |- | --groupId || '''-g''' || group comp object id |- | --voId || '''-v''' || vo comp object id |- | --resourceId || '''-r''' || resource comp object id |- | --serviceId || '''-s''' || service comp object id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== updateUser ==== Updates user. User id is required. -------------------------------------- Available options: {| width=75% | --userId || '''-u''' || user id |- | --firstName || '''-f''' || first name for update |- | --lastName || '''-n''' || family name for update |- | --middleName || '''-d''' || middle name for update |- | --titleBefore || '''-t''' || title used before the name for update |- | --titleAfter || '''-T''' || title used after the name for update |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== updateGroup ==== Updates group. Group id is required. -------------------------------------- Available options: {| width=75% | --groupId || '''-g''' || group id |- | --name || '''-n''' || new name for the group |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== updateNotifReceiver ==== Updates NotifReceiver. Id is required field. -------------------------------------- Available options: {| width=97% | --NotifReceiverId || '''-i''' || id of the NotifReceiver |- | --target || '''-r''' || target |- | --typeOfReceiver || '''-t''' || type of receiver (EMAIL_USER/EMAIL_GROUP/JABBER) |- | --templateId || '''-p''' || template id (template has to exist) |- | --locale || '''-l''' || locale |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== updateNotifRegex ==== Updates a NotifRegex. ID is required field. -------------------------------------- Available options: {| width=57% | --NotifRegexId || '''-i''' || id of the NotifRegex |- | --regex || '''-r''' || regex |- | --note || '''-n''' || note |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== updateNotifTemplate ==== Updates a NotifTemplate. Id of the updated Template is required field. -------------------------------------- Available options: {| width=82% | --id || '''-i''' || id of the updated NotifTemplate |- | --name || '''-n''' || name |- | --primaryProperty || '''-p''' || primary property (key and value divide by '=', |- | || || values divide by '/', more properties allowed) |- | --locale || '''-l''' || locale |- | --sender || '''-s''' || sender |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== updateNotifTemplateMessage ==== Updates a NotifTemplateMessage. Id is required field. -------------------------------------- Available options: {| width=55% | --NotifTemplateMessageId || '''-i''' || NotifTemplateMessage id |- | --templateId || '''-t''' || template id |- | --locale || '''-l''' || locale |- | --message || '''-m''' || message |- | --subject || '''-s''' || subject |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== changeVoMemberStatus ==== Change a member's status. Member id is required. -------------------------------------- Available options: {| width=65% | --memberId || '''-m''' || member id |- | --newStatus || '''-s''' || new status |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== copyForm ==== Coppies Form from VO to Group or from Group to VO ------------------------------------------------- Available options: {| width=90% | --fromVo || '''-v''' || from VO id |- | --toGroup || '''-G''' || to Group id |- | --fromGroup || '''-g''' || from Group id |- | --toVo || '''-V''' || to VO id |- | --notifMails || '''-m''' || copy with notification mails |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ------------------------------------------------- At first empty Form for target group or VO must be created !!! ==== copyRegMails ==== Coppies Registration Nitification mails from VO to Group or from Group to VO ------------------------------------------------- Available options: {| width=65% | --fromVo || '''-v''' || from VO id |- | --toGroup || '''-G''' || to Group id |- | --fromGroup || '''-g''' || from Group id |- | --toVo || '''-V''' || to VO id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ------------------------------------------------- At first empty Form for target group or VO must be created !!! ==== forceServicePropagation ==== Forces service propagation. Facility and service are required fields. -------------------------------------- Available options: {| width=60% | --serviceId || '''-s''' || service id |- | --serviceName || ''' -S''' || service name |- | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== forceGroupSynchronization ==== Synchronize the group with the external source. ------------------------------------ Available options: {| width=45% | --groupId || '''-g''' || group id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== doTheMagic ==== Sets all atributes for the member to valid value. ------------------------------------ Available options: {| width=55% | --memberId || '''-m''' || member id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfAuditerMessages ==== Prints list of auditer messages --------------------------- Available options: {| width=110% | --limit n || '''-l''' || n limit results to the n number of lines |- | || ||(20 by default) |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfMembersBySearcher ==== Prints list of members found by searching through user attributes. Need to specify vo where will be members looking for. --------------------------- Available options: {| width=100% | --voId || '''-v''' || VO idetifier |- | --voShortName || '''-V''' || VO short name |- | --mapOfAttrs || '''-m''' || Map with attr_names and values for searching |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfPublications ==== Prints list of publications. --------------------------- Available options: {| width=80% | --orderById || '''-i''' || order by identifier of publication |- | --orderByName || '''-n''' || order by title of publication |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== getTaskResultsForDestinations ==== Returns state of the GEN and SEND services on the defined destinations --------------------------------------------------------------- Available options: {| width=80% | --destinations || '''-d''' || destinations separated by space |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== makeUserPerunAdmin ==== Adds PERUNADMIN role to the user. User id is required. ----------------------------------------------------- Available options: {| width=72% | --userId || '''-u''' || user id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== loadExtSourcesDefinitions ==== Loads external source definitions. Tool can be used only by PERUNADMIN. It is target for updating of external sources from perun configuration. ----------------------------------- Available options: {| width=90% | --help || '''-h''' || prints this help |} ==== startNotifications ==== Starts notifications ------------------------------------ Available options: {| width=80% | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== stopNotifications ==== Stop notifications ------------------------------------ Available options: {| width=80% | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfAllFacilitiesGroupContacts ==== Prints list of Facilities Contacts --------------------------------------------- Available options: {| width=80% | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== addFacilityContacts ==== Adds contacts to facility. ------------------------------------ Available options: {| width=60% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --contactName || '''-n''' || contact group names |- | --owners || '''-o''' || list of owner's Id |- | --groups || '''-g''' || list of group's Id |- | --users || '''-u''' || list of user's Id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeFacilityContacts ==== Removes contacts from facility. ------------------------------------ Available options: {| width=60% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --contactName || '''-n''' || contact group names |- | --owners || '''-o''' || list of owner's Id |- | --groups || '''-g''' || list of group's Id |- | --users || '''-u''' || list of user's Id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== getContactsOfFacility ==== Lists contacts of facility. ------------------------------ {| width=72% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --orderByName || '''-n''' || order by contact name |- | --displayIds || '''-i''' || display Id of owner/user/group |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== getContactsAssignedToGroup ==== Lists contacts of group. ------------------------------ {| width=62% Available options: | --groupId || '''-g''' || group's Id |- | --groupName || '''-G''' || group name |- | --voId || '''-v''' || vo id |- | --voShortName || '''-V''' || vo short name |- | --orderByName || '''-n''' || order by contact name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== getContactsAssignedToOwner ==== Lists contacts of owner. ------------------------------ {| width=62% Available options: | --ownerId || '''-o''' || owner's Id |- | --ownerName || '''-O''' || owner's Name |- | --orderByName || '''-n''' || order by contact name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== getContactsAssignedToUser ==== Lists contacts of user. ------------------------------ Available options: {| width=62% | --userId || '''-o''' || user's Id |- | --orderByName || '''-n''' || order by contact name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfExpiredMembers ==== Prints list of expired V OMembers. Vo and number of years are required fields. --------------------------------------------------- Available options: {| width=82% | --voId || '''-v''' || VO idetifier |- | --voShortName || '''-V''' || VO short name (default meta) |- | --expiredYears || '''-y''' || numer of years (how long is expired) |- | --expired || '''-e''' || expired members only (default) |- | --expAndDis || '''-d''' || expired and disabled members |- | --orderById || '''-i''' || order by Member's identifier |- | --orderByName || '''-n''' || order by contact name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== addSecurityTeamManager ==== Adds user or group as SecurityTeam admin. User id or group id and SecurityTeam are required fields. ------------------------------------ Available options: {| width=52% | --securityTeamId || '''-s''' || SecurityTeam id |- | --userId || '''-u''' || user id |- | --groupId || '''-g''' || group Id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== assignSecurityTeamToFacility ==== Assigns SecurityTeam to facility -------------------------- Available options: {| width=52% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --securityTeamId || '''-s''' || SecurityTeam id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== createSecurityTeam ==== Creates a securityTeam. SecurityTeam name is required field. -------------------------------------- Available options: {| width=75% | --securityTeamName || '''-S''' || securityTeam name (no space allowed) |- | --description || '''-d''' || description |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== deleteSecurityTeam ==== Deletes a securityTeam. SecurityTeam id is required field. -------------------------------------- Available options: {| width=72% | --securityTeamId || '''-s''' || Id of deleted securityTeam |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== getSecurityTeamDetails ==== Displayes Admins and BlackList of SecurityTeam. SecurityTeam is required field. ------------------------------------ Available options: {| width=72% | --securityTeamId || '''-s''' || securityTeam id |- | --orderById || '''-i''' || order by user's identifier |- | --orderByName || '''-n''' || order by user's name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== getSecurityTeamManagers ==== Displayes list of SecurityTeam admins. SecurityTeam is required field. ------------------------------------ Available options: {| width=77% | --securityTeamId || '''-s''' || securityTeam id |- | --orderById || '''-i''' || order by manager's identifier |- | --orderByName || '''-n''' || order by manager's name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== getSecurityTeamsOfFacility ==== Lists SecurityTeams assigned to facility. ------------------------------------ Available options: {| width=77% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --orderById || '''-i''' || order by securityTeam id |- | --orderByName || '''-n''' || order by securityTeam name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== listOfSecurityTeams ==== Prints list of all Security Teams --------------------------------------- Available options: {| width=80% | --orderByName || '''-n''' || order by securityTeam name |- | --orderById || '''-i''' || order by securityTeam id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeSecurityTeamFromFacility ==== Removes SecurityTeam from facility -------------------------- Available options: {| width=60% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --securityTeamId || '''-s''' || SecurityTeam Id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeSecurityTeamManager ==== Removes user or group from SecurityTeam admins. User id or group id and SecurityTeam are required fields. ------------------------------------ Available options: {| width=60% | --securityTeamId || '''-s''' || SecurityTeam id |- | --userId || '''-u''' || user id |- | --groupId || '''-g''' || group Id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== updateSecurityTeam ==== Updates a securityTeam. SecurityTeam id is required field. -------------------------------------- Available options: {| width=85% | --securityTeamId || '''-s''' || Id of updated securityTeam |- | --securityTeamName || '''-S''' || new securityTeam name (no space allowed) |- | --description || '''-d''' || new description |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== addUserToBlacklist ==== Adds user to BlackList SecurityTeam. User id and SecurityTeam are required fields. ------------------------------------ Available options: {| width=65% | --securityTeamId || '''-s''' || SecurityTeam id |- | --userId || '''-u''' || user id |- | --desc || '''-d''' || purpose of blacklisting |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== getFacilityBlackList ==== Displayes BlackList at Facility. Facility Id or name are required fields. ------------------------------------ Available options: {| width=77% | --facilityId || '''-f''' || facility id |- | --facilityName || '''-F''' || facility name |- | --orderById || '''-i''' || order by user's identifier |- | --orderByName || '''-n''' || order by user's name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== getSecurityTeamBlackList ==== Displayes BlackList of SecurityTeam. SecurityTeam is required field. ------------------------------------ Available options: {| width=70% | --securityTeamId || '''-s''' || SecurityTeam id |- | --orderById || '''-i''' || order by user's identifier |- | --orderByName || '''-n''' || order by user's name |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ==== removeUserFromBlackList ==== Removes user from BlackList SecurityTeam. User id and SecurityTeam are required fields. ------------------------------------ Available options: {| width=65% | --securityTeamId || '''-s''' || SecurityTeam id |- | --userId || '''-u''' || user id |- | --batch || '''-b''' || batch |- | --help || '''-h''' || prints this help |} ---------------------------------- === How to propagate changes === To propagate a change immediatelly, use tool: [[#forceServicePropagation|forceServicePropagation]]. Changes will be inserted into queue and reaching head of queue, propagated to machines. === Perun RPC servlet === ==== URL ==== <servlet address Perun-RPC>/serializer/manager/method * serializer: only 'json' at the moment * manager: name of manager Perun-API, e.g. 'vosManager', 'membersManager'... * method: name of method of particular manager e.g. 'getVos'... Example: https://alcor.ics.muni.cz/krb/rpc/json/vosManager/getVos ==== Parameters of methods ==== * Parameters are passed in the URL address (HTTP GET): <servlet address Perun-RPC>/serializer/manager/method?parametr1=value1&parametr2=value Only methods that do not change the state (get* methods etc.) and only with parameters with type string, int or bean already existing in database (in this case expected value is bean's id). Example: https://alcor.ics.muni.cz/krb/rpc/json/membersManager/getMembers?vo=3&pageSize=20&pageNum=1 * in format JSON (HTTP POST) ==== Format of response ==== Response format depends on serializer, now works JSON format. Beans are serialized to JSON objects, objects List to JSON arrays and other types to the responding types in format JSON. ==== Exceptions ==== Exceptions are thrown as JSON object with value 'errorId' (id of error), 'type' (type of error) and 'errorInfo' (details of error). Concerning exception of Perun API, value 'type' match with class name containing this exception and 'errorInfo' is result of method getMessage() on this exception. <br/> <hr/> Správa projektu je dostupné na uzavřeném Tracu https://projekty.ics.muni.cz/perunv3/trac. [[Kategorie:Perun]] [[Kategorie:English]] 8q92q27dn2yxhmygnzvsqlu2klb45yl 0 108 332 331 2016-01-23T15:48:53Z Root 1 naimportována 1 revize wikitext text/x-wiki {{Template:Perun-en-template}} [[My role is VO MANAGER|''VO MANAGER]] ''role needed'' {{info|There is also a [[Create_application_form|short, more straightforward version of documentation.]]}} == Application workflow == [[File:Application_workflow.png|left|alt=Application workflow.|Application workflow.]] * User display a form ** he is not a member - first form s displayed ** he is already member -- application to extend a membership is displayed ** application has prefilled items with data taken from perun and federation (perun data has precedence over federation) * User submits an application form (NEW state) ** application data is saved in DB, KDC logins are reserved ** VO manager receives a notification when new application form is submitted ** user receives a notification when he submits an application form * Email verification ** e-mail is not required in application form or it is prefilled from federation data *** application is verified automatically (move to state VERIFIED) ** user sets new email (different from the one from federation) *** email notification with link to email verification is send to user *** when user click on the verification link, application is verified (move to state VERIFIED) * Approval and rejection of application ** manager approves (state APPROVED) or rejects (state REJECTED) application in GUI ** approval *** first application form - creates new VO member *** application to extend a membership - extends a membership in VO *** notification sent to user via email about approval *** after approval, no more first (INITIAL) form can be filled by the user ** rejection *** KDC login reservation is removed *** notification sent to user via email about rejection *** user can submit new first (INITIAL) application === VO application form === * every VO has only one form with manual or automatic approval (it can be set) * division into first (INITIAL) and extension (EXTENSION) application form is based on items in the form. Each item has set whether is shown in the first form or in the extension form or in both. * if user is not VO member, first form is displayed. Otherwise, extension form is displayed. * until first application is approved, user can not submit another form. Extension applications can user submit as many as he needs. * membership rules in VO can restrict possibility to submit application form in VO (e.g. VO attribute membershipExpirationRules requires concrete user LoA, assigns period extension or length of membership). ==== Group application form ==== * every group in VO can have unique application form (it must be created in Group manager section). * when first application to group is submitted, not being member of parental VO user automatically submits a first application to the VO * after approval, user is automatically incorporated into the group. * as far as groups are concerned, extension application form can not be submitted (membership lasts until member is in parental VO or until member is manually removed from group) * group application form has own email notification different from VO's notification (to have both same, duplicate notification must be created) ==== Attributes in VO application form ==== Attributes must be set on the page ''VO manager - Settings'' (Add button). Same attributes are required in application to groups. * '''Email address "From"''' - email address used as “from” for all email notification to VO or group users. * '''Email address "To"''' - a mailing list used as “to” for all email notification to VO or group manager. * '''URL logo VO''' - URL address (including http://) to picture with VO logo (it would be shown in the top of application form) ==== Modules ==== * Module that do additional action with new user (add into group, set default attribute values, etc.) can be programmed to every application form (both VO and group). * Module is called after three actions: submitting, approval or rejection of application are successfully finished. * Name of module class is set in GUI in application configuration (button ''Settings''). ==== Automatic approval ==== [[File:Prihl_auto_schvalovani.png|upright=1.5|center|alt=Automatic approval.|Automatic approval.]] * If approval is automatic, it is done right after submitting of application by user. Verification is required; therefore it may wait to manual verification of email by user. * Automatic approval is divided into first application form and extension application form. ==== Redirecting from application form ==== 3 params ("&param=text") to redirect could be inserted into URL of application form. Parameter value is URL of web page to where redirect. '''targetnew''' - Redirecting runs automatically after sending and receiving new (INITIAL) form. When error occurred, redirecting can not proceed, error alert appears on the page. '''targetextended''' - Redirecting runs automatically after sending and receiving (EXTENSION) application form. When error occurred, redirecting can not proceed, error alert appears on the page. '''targetexisting''' - Redirecting runs automatically after form loading, if user is already VO/group member and it is not a period to extend a membership (VO attribute ''Membership expiration rules '', item ''gracePeriod'' - sets number of months before expiration date when user can extend his membership. == Work with items in the form == ''' All changes in form are just temporary and are saved only after pressing ''Save''''' in the main bar. '''''Save'' rewrites the form as a whole in the DB! When error occurred, no change is saved!''' === How to add an item in a form === * button ''Add'' in the main bar solves this task. Creation has two phases and newly created items are green-colored in the example. [[File:Prihl_add_form_item.png |upright=1.5|center|alt=Add items.|Add items.]] ==== Item atrributes creation ==== '''Short name''' - is '''compulsory attribute''' to identify the item. If no description is added, short name is listed in the form. '''Input type''' - sets the type of input. Detailed description is below. '''Insert after''' - insert new item after another item or on the very beginning of form. [[File:Prihl_create_form_item.png|upright=1.5|center|alt=Add items.|Add items.]] ==== Item types ==== '''Custom HTML text''' - Any HTML-formatted text, e.g. general instructions for application form. Content is saved into parameter label depending on language. '''Custom submit button''' - Submit button in the form. Label sets text in the button. '''Submit button with autosubmit''' - If input in a form is valid, it's automatically submitted without user interaction. '''Input text field''' - free short text (max. 512 characters) '''Input text multi-line field''' - free long text (max. 1024 characters). '''Single value selection from list''' - PullDown menu, only one answer per user '''Single value selection with opt. custom value''' - PullDown menu with option “- other value -” to enable inserting other value (as textfield) by user. '''Input text pre-filled from external source''' - uneditable text field containing values taken from federation '''Hidden input text pre-filled from external source''' - hidden text field containing values taken from federation, hidden for user but visible for VO manager. It serves to save data taken from federation. '''Input text field for email''' - editable field to set an email to verification. All notifications are sent to this email. When is filled value same as the verified one, email is consider verified. '''Input text field for username / password''' - editable fields, but when USERNAME is prefilled, it can not be changed and field PASSWORD is hidden. '''The relationship between both items is created by value of parameter ''Perun destination attribute'' or ''federation attribute (but just when filled with data from federation)'''''. Attribute name is connected with login in namespace or parameter name from federation, e.g. ''urn:perun:user:attribute-def:def:login-namespace:einfra'' or ''krbPrincipalName''. Different logins for different namespace are allowed, but it is necessary to keep a relationship via parameter and order of items. If login is empty in the form and it is typed by user, dynamic control of login availability check it. '''Selection of timezone''' - Selection box pre-filled with available timezones like: Europe/Prague '''Header''' - Custom text displayed as heading. === Configuration === Common parameters are: '''Required''' - Item is required and user must fill it. Asterisk after item's name marks the item in the form. '''Regular expression''' - if filled, item value is checked with regexp in field. '''Federation attribute''' - connection of item to data from federation. Item is pre-filled if data from federation is available. Allowed values are: * ''displayName'' (displayed name), * ''cn'' (common name), * ''mail'' (email address), * ''o'' (organisation), * ''loa'' (level of assurance), * ''eppn'' (mail?), * ''givenName'' (given name), * ''sn'' (surname). '''destination attribute''' - relation to attribute in Perun system. Value submitted by user is saved to the particular attribute and pre-fills items in application. Only attributes for entities ''user'' and ''member'' can be saved and pre-filled. '''Use in application types''' - sets which type of application (first or extension) contains an item [[File:Prihl_edit_form_item.png|upright=1.5|center|alt=Edit an item.|Edit an item.]] Language bookmark: '''Label''' - main description shown to user. When missing, Short name is used instead. '''Help''' - text shown as help for an item. '''Error''' - text that appears when item value does not match with '''Regular expression'''. '''Box contents''' - predefined values for items with type SELECTIONBOX a COMBOBOX that user can select. Value saved in DB and description are specified. [[File:Prihl_edit_form_item_lang.png|upright=1.5|center|alt=Add an item.|Add an item.]] === How to delete, edit and order items === Item is deleted by button ''Delete'' placed in the line with item definition. Item is marked to remove, but removed is when form is saved (by click on ''Save''). Until the form is saved, button ''Undelete'' reverse the process. Before you delete an item, consider a possibility simply just to hide an item from both types of form (first and extension). Order of items are defined in the overview of all items. Order can be changed by buttons with arrows. Button ''Edit'' enables to edit of all available parameters, editing is confirmed by button ''OK''. All changes are saved after button ''Save'' is pressed. === Application overview === Button ''Preview form'' serves to see an overview from user's point of view. Overview use cached information so to see changes in form, overview must be closed and reopen. Overview implements validation items with regexp, but functionalities to send a form or to pre-fill item automatically do not work. Overview supports switching between type (INITIAL, EXTENSION) and language of form (Czech, English). == E-mail notification == Button ''E-mail notifications'' serves to configure email notifications. E-mail definitions for specified actions during registration process can be added, supporting also more language mutations, can be switched on/off and special variables serves to substitute them with actual data depending on user and his application form. When error occurred, mail is not send and error is logged. [[File:Prihl_email_notify.png|upright=1.5|center|alt=Email notifications.|Email notifications.]] ==== Language ==== Notification language is set by any item of form with relationship to attribute ''urn:perun:user:attribute-def:def:preferredLanguage''. When this item is missing and user has an account in Perun, value is taken from his attribute, otherwise English version is used. ==== Notification type ==== Definition must be created for first and extension application separately. When definition does not exist, mail is not send. APP_CREATED_USER - confirmation of submitting application form by user. It is send to the user to the first email address inserted into field VALIDATED_EMAIL. When address or array do not exist in the form, mail preferred by user is used. APP_CREATED_VO_ADMIN - notification to VO manager about new application. It contains link to application's detail to approve or deny it. When during process errors occurred, list of errors is attached. These errors can be sent to solve by perun admin. MAIL_VALIDATION - notification containing link to validation to check an email address. This email is sent just in case unapproved email is in array VALIDATED_EMAIL (its value is different from pre-filled). APP_APPROVED_USER - notification to user that his application was accepted and he became VO member (his membership was extended). APP_REJECTED_USER - notification to user that his application was rejected by VO manager. During rejection in GUI, reason of rejection can be inserted and consequently sent as part of notification. ==== Dynamic content substitution ==== There are several keywords to use in email to substitute with actual data. These keywords can be used in both email subject and text with only one exception '''{validationLink}''', which can be used only in email text with type MAIL_VALIDATION ! When keyword misses a value or error occurred, free space is inserted instead. *'''{appId}''' - application number *'''{actor}''' - login of user who submit an application (login used to logn in GUI) *'''{extSource}''' - external resource from where user comes (used to logn in GUI) *'''{firstName}''' - first name of user related to form item with perun destination attribute ''urn:perun:user:attribute-def:core:firstName'' *'''{lastName}''' - last name of user related to form item with perun destination attribute ''urn:perun:user:attribute-def:core:lastName'' *'''{displayName}''' - full user name with academic degrees related to form item with perun destination attribute ''urn:perun:user:attribute-def:core:displayName'' *'''{login-[namespace]}''' - user login in the namespace related to form item with type USERNAME with perun destination attribute ''urn:perun:user:attribute-def:def:loginNamespace:[namespace]''. [namespace] can be substituted with values: ''einfra'', ''meta'', ''egi-ui'', ''cesnet'', ''mu''. E.g.: {login-einfra} *'''{membershipExpiration}''' - expiration date after creation or extension VO membership. It is taken from already existing user so it is not necessary to be in application form. It is related to member attribute ''urn:perun:member:attribute-def:def:membershipExpiration''. *'''{voName}''' - full name of VO to which application belongs *'''{groupName}''' - full name of group to which application belongs *'''{mailFooter}''' - general mail footer used for emails for VO, related to value of attribute ''urn:perun:vo:attribute-def:def:mailFooter'' that should be set only in GUI interface in order not to lost formating *'''{appGuiUrl}''' - link to register GUI of Perun system *'''{appDetailUrlFed}''', '''{appDetailUrlKerb}''', '''{appDetailUrlCert}''' - links to details of application form with different type of authentication to be used by VO manager *'''{perunGuiUrlFed}''', '''{perunGuiUrlKerb}''', '''{perunGuiUrlCert}''' - link to Perun GUI with different type of authentication *'''{errors}''' - List of errors that occurred during whole process. Now only sent in notification to VO manager about application creation. *'''{customMessage}''' - customizable message to add to specified type of notification to user. '''At the moment used only in notification to user about application rejection''' (it contains specific reason written by VO manager in GUI during process of rejection). [[Category:Perun]] [[Category:Application form]] [[Category:English]] 4gjqyn5d6nwbb9il465iw18kllpsppz 0 109 334 333 2016-01-23T15:48:53Z Root 1 naimportována 1 revize wikitext text/x-wiki {{Template:Perun-en-template}} == Slave scripts == * Gen script creates file .tar.gz with data for end services and for files '''VERSION''' and '''SERVICE'''. File '''VERSION''' contains version of gen scripts checked by slave script. File '''SERVICE''' contains service's name. According to this name end machine selects service slave script. * End machine runs script'''/opt/perun/bin/perun'''. This script decide which script should be run (depending on content file SERVICE). * As a transport layer SSH is used. Using SSH key, only '''/opt/perun/bin/perun''' is set to run at the end machine. === Debian package repository === ==== Template sources.list==== Template is stored in <code>/afs/ics.muni.cz/common/etc/sources.list.lenny</code> ==== Repostory for MetaCentrum ==== Location: '''depot1.mc.cesnet.cz'''<br /> Configuration APT sources.list: <nowiki>ftp://depot1.mc.cesnet.cz/</nowiki> all main <nowiki>ftp://depot1.mc.cesnet.cz/</nowiki> [lenny|squeeze] main Following key (PGP PUBLIC KEY BLOCK) must be downloaded and added into trustworthy resources by following command: apt-key add /tmp/key PGP key: '''C3C8B519''' <pre> -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v1.4.6 (GNU/Linux) mQINBEvVSjABEACo7dd0akbgM+C+Oph64KHYaF2Cezsv2Ngc2W/OGZ3dhCdhxbE/ 7dnt4Mm5V5eLzuevgf90Pm/W1k0AAlYPqDmiHlom45G1J+XrQqWhZNPv7HCiAj/X 7tiXV/Gp4BfQvJJasilAACTkqbsloANRktd9S1k5jRd+zfVvkNEoEUW9/HT6w6Z5 ZRlYixw/ooDpcX4uK7rHeTtC0udLDwAWY281/zn8XMPEvLo7ql+5kABJIy6iZJ2o vyWyo4SwYkYOHPcni4Cy6jCGP9LZR61sweOcsdfc8vsvr120OdFuTpR9X9gm6K20 tX8PDEy3GzMreVtrI/bJrcVbu+oz7cCycl+8qIkNtX+B2zC7tslE316xfoat7ZIb sYQcHXTlvedfMS7NtZ8NfOVernwt3tWffBmyTSrmlrqTGOwes6Vm2xhXZ7/h9K+W 7zEFTID8idpHqnDdx9DFFuUeQ6IcmAOjE4Xny/bfw0jan3/0+Ncv1FX5NJzf7GdH 4Xm85v2DNA689jHziJv3X/QLKtP4LEA0JmZD++9hAMd5XJ1lobSJZqytHlOKPjGg /eSwBaVgHENbEeHBMAET3QL5J1cFzUqS3HXrCoWh8MSoq3XYLPtLxZrSEX8z5WKh pE5FLx0FGSi5MFyHg2WqBDkqSTN3Doe1uh8SoT9vVFuPb1m4cAR1KzPGHwARAQAB tFhNZXRhQ2VudHJ1bSBQYWNrYWdlIFJlcG9zaXRvcnkgKE1ldGFDZW50cnVtIFBh Y2thZ2UgUmVwb3NpdG9yeSBQR1Aga2V5KSA8bWV0YUBjZXNuZXQuY3o+iQI8BBMB AgAmAhsDBgsJCAcDAgQVAggDBBYCAwECHgECF4AFAlF5lz0FCQtH540ACgkQVc75 6MPItRlf1Q/+IonUahNhAYQLhkdZvIPyT099KBviqMYXs1DJO940wIfX26ijz/v0 fiEWvD1TlCx7xmegUj0u8EoB4TE4DYl1cPUZyQF+B2m9dLBT7umEypvZpuHjcBZe LpBL2K04gJYtiDe8yMTWChlrg+gcSaF+FkB/K9YYyjlkfIXp15WHSlNXN+aiB/3P 8GMJiRPU0g3ScnwBfrLAXUX8stlBFzk2OVcrWmXQoHha/1cEn7w8JEEN4dOQIuKs Y7rItaS80HFpfwP10cU/l6ohMOh5cpf36qWPVKsez+wgeO8ah/7ZOtEG3QTktk6x bWzSGJ55beYm88iBvQuYJ6Xk8cpXsuFmaRED09mvXvoRYhUKovt+m7W8dW1s4h0y /x3ER7jPiUSdGkepag1J+WQtzrsSPgWLt5x8C026iQcvK6e72lbLDSX9cA8QdPAM Vnc5cPF4Jxz1lW0OHKKW5nMKPUTp/YZMfZcQTM2rkpLEZHAdC3WgbnM3N+gGY3vN qpBTPoFAWVZTsM9BQ8A/bJJBgDXg0SSnChHe4hxwQJuXR+tAO8OSk2z8xCdeRZqK 9WAMviqmYvoCZMI4F/QJfKlSAoHrfhQPWOo2iS3aKUlBJgWxaejiJAwNwKDujnU5 F0Y8sXypxJudVbi6/Q9BE/tGOTKwtaFiB5Gon+mGrvvnJWTGe7VjzriIRgQQEQIA BgUCS9VRAAAKCRCKeUuOc6DkdYCfAJ9MhgOfaAlSRnaSfZ7sfzrIept/DQCfXiGM kO6S6OD1WngSJiCst3UTkW4= =nY1A -----END PGP PUBLIC KEY BLOCK----- </pre> ===== Directory structure ===== /storage/software/repository `--bin `--Makefile |--gpg `--PGP files `--ftp (generated directory) `--dists |--pool `--packages `--all `--main `--etch `--main `--lenny `--main `--sarge `--main ===== New package installation into repository ===== # Copy deb package into particular directory into '''packages''' # Run <code>make</code> in directory '''bin''' (must be done at smaug1 machine) # Installation of new packages requires PGP key password. ===== How to create a deb package ===== Debian package can be created in two ways. First are tools '''dh_make''' and '''dpkg-buildpackage'''. Tutorials are available here:<br /> [http://www.abclinuxu.cz/clanky/navody/balicky-pro-debian-1-uvod in detail in czech]<br /> [http://www.webupd8.org/2010/01/how-to-create-deb-package-ubuntu-debian.html brief in english]<br /><br /> Second way is to use script '''checkinstall'''. This script is easier, but not so flexible as previous solution. Briefly described in [http://penguininside.blogspot.com/2009/12/how-to-build-deb-package-from-targz.html tutorial]. Tutorial to edit already existing packages and their maintenance with git is [https://erebor.ics.muni.cz/wiki/git.html here]. == LDAP == * Information about groups are exported into LDAP. * Data is pushed via LDAPc component into LDAP which use Auditer. Delay in communication takes a couple of seconds. * Inner structure schema: dc=perun,dc=cesnet,dc=cz | ------------------- | | perunVoId=voId ou=People | | perunGroupId=groupId perunUserId=userId ... == LDAP - related schemas == * In directory /etc/ldap/schema following schemas are required: core.schema inetUser.schema tenOperEntry.schema perun.schema nis.schema openldap.schema inetorgperson.schema cosine.schema * Current perun.schema ** Attributes <pre>{0}( 1.3.6.1.4.1.8057.2.80.1 NAME 'perunUserId' DESC 'Unique user identifier managed by Perun' EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE )</pre> <pre>{1}( 1.3.6.1.4.1.8057.2.80.16 NAME 'perunResourceId' DESC 'Unique resource identifier managed by Perun' EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE )</pre> <pre>{2}( 1.3.6.1.4.1.8057.2.80.17 NAME 'perunFacilityId' DESC 'Unique facilityId identifier managed by Perun' EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE )</pre> <pre>{3}( 1.3.6.1.4.1.8057.2.80.2 NAME 'perunGroupId' DESC 'Unique group identifier managed by Perun' EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE )</pre> <pre>{4}( 1.3.6.1.4.1.8057.2.80.3 NAME 'perunVoId' DESC 'Unique VO identifier managed by Perun' EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE )</pre> <pre>{5}( 1.3.6.1.4.1.8057.2.80.12 NAME 'isServiceUser' DESC 'If user is Service User then 1, else 0' EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE )</pre> <pre>{6}( 1.3.6.1.4.1.8057.2.80.11 NAME 'assignedToResourceId' DESC 'Identifier of resource which is group assigned to.' EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 )</pre> <pre>{7}( 1.3.6.1.4.1.8057.2.80.18 NAME 'assignedGroupId' DESC 'Identifier of group which is assigned to this resource' EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 )</pre> <pre>{8}( 1.3.6.1.4.1.8057.2.84.7 NAME 'preferredMail' DESC 'RFC1274: RFC822 Mailbox' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26{256} )</pre> <pre>{9}( 1.3.6.1.4.1.8057.2.80.9 NAME 'login' DESC 'User login for namespace' EQUALITY caseIgnoreIA5Match SYNTAX 1.3.6.1.4.1.1466.115.121.1.26{256} )</pre> <pre>{10}( 1.3.6.1.4.1.8057.2.80.14 NAME 'perunUniqueGroupName' DESC 'uniqueName of group like "VoName:ParentGroupName:GroupName"' EQUALITY caseIgnoreMatch SUBSTR caseIgnoreSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE )</pre> <pre>{11}( 1.3.6.1.4.1.8057.2.80.10 NAME 'eduPersonPrincipalNames' DESC 'User ExtLogin for UserExtSource with Type IDP' EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 )</pre> <pre>{12}( 1.3.6.1.4.1.11433.2.2.1.9 NAME 'userCertificateSubject' SUP name )</pre> <pre>{13}( 1.3.6.1.4.1.8057.2.80.8 NAME 'perunParentGroupId' DESC 'identifier (perunGroupId) of the parent group as managed by Perun' EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 SINGLE-VALUE )</pre> <pre>{14}( 1.3.6.1.4.1.8057.2.80.13 NAME 'perunParentGroup' DESC 'Dn of the parentGroup' EQUALITY distinguishedNameMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.12 )</pre> <pre>{15}( 1.3.6.1.4.1.8057.2.80.20 NAME 'libraryIDs' DESC 'IDs of library and users in external library system' EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 )</pre> <pre>{16}( 1.3.6.1.4.1.8057.2.80.19 NAME 'memberOfPerunVo' DESC 'Identifier of vo where user has member' EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 )</pre> **Objects <pre>{0}( 1.3.6.1.4.1.8057.2.80.4 NAME 'perunUser' DESC 'User managed by Perun' SUP inetOrgPerson STRUCTURAL MUST ( perunUserId $ isServiceUser ) MAY ( preferredMail $ userCertificateSubject $ uidNumber $ login $ eduPersonPrincipalNames $ userPassword $ memberOfPerunVo $ libraryIDs) )</pre> <pre>{1}( 1.3.6.1.4.1.8057.2.80.5 NAME 'perunGroup' DESC 'Group managed by Perun' SUP top STRUCTURAL MUST ( cn $ perunGroupId $ perunVoId $ perunUniqueGroupName ) MAY ( uniqueMember $ businessCategory $ seeAlso $ owner $ ou $ o $ description $ perunParentGroup $ perunParentGroupId $ assignedToResourceId ) )</pre> <pre>{2}( 1.3.6.1.4.1.8057.2.80.15 NAME 'perunResource' DESC 'Resource managed by Perun' SUP top STRUCTURAL MUST ( cn $ perunResourceId $ perunVoId $ perunFacilityId ) MAY (uniqueMember $ businessCategory $ seeAlso $ owner $ ou $ o $ description $ assignedGroupId ))</pre> <pre>{3}( 1.3.6.1.4.1.8057.2.80.6 NAME 'perunVO' DESC 'VO managed by Perun' SUP organization STRUCTURAL MUST perunVoId MAY uniqueMember )</pre> === Installation === * In slapd.conf change: suffix "dc=perun,dc=cesnet,dc=cz" rootdn "cn=admin,dc=perun,dc=cesnet,dc=cz" rootpw {SSHA} (generated slapdpassed) access to attrs=userPassword,shadowLastChange by dn="cn=admin,dc=perun,dc=cesnet,dc=cz" write by anonymous auth by self write by * none access to * by dn="cn=admin,dc=perun,dc=cesnet,dc=cz" write by * read attributeoptions x-ns- * ldapadd -f init-perun.ldif -x -D cn=admin,dc=perun,dc=cesnet,dc=cz -W * Example of schema # Structure: # dc=perun,dc=cesnet,dc=cz # - ou=People (organizationalUnit) # - perunId=<userId> (perunPerson) # - o=VO1 (perunOrganization) # - cn=Group1 (perunGroup) # - cn=Group2 (perunGroup) # - o=VO2 (perunOrganization) # - cn=Group1 (perunGroup) # Examples: # A perunPerson dn: perunUserId=1234,ou=People,dc=perun,dc=cesnet,dc=cz objectclass: top objectclass: person objectclass: organizationalPerson objectclass: inetOrgPerson objectclass: perunUser objectclass: tenOperEntry objectclass: inetUser entryStatus: active|dead sn: surname cn: name surname givenName: name perunId: 1234 mail: xx@yy.zz isServiceUser: 0 preferedMail: aaa@bbb.ccc userCertificateSubject: \D userCertificateSubject: \E eduPersonPrincipalNames: aaa@bbb.cde o: Organisation from IdP uidNumber;x-ns-cesnet: 1523 uidNumber;x-ns-lhota: 123 login;x-ns-metacentrum: abbc login;x-ns-lhota: bbca memberOfPerunVo: vo1 memberOf: perunGroupId=123456,perunVoId=12345,dc=perun,dc=cesnet,dc=cz libraryIDs: 123 libraryIDs: 345 # A perunVO dn: perunVoId=12345,dc=perun,dc=cesnet,dc=cz objectclass: top objectclass: organization objectclass: perunVO o: testVO1 (shortName) perunVoId: 12345 description: Testovaci VO 1 (longName) # A perunGroup dn: perunGroupId=123456,perunVoId=12345,dc=perun,dc=cesnet,dc=cz objectclass: top objectclass: perunGroup cn: Group1 (Group name in Perun) perunUniqueGroupName: VO:ABC:Group1 description: group description perunGroupId: 123456 perunParentGroup: ABC perunParentGroupId: 12345 assignedToResourceId: 12345 uniqueMember: perunUserId=1234,ou=People,dc=perun,dc=cesnet,dc=cz uniqueMember: perunUserId=98765,ou=People,dc=perun,dc=cesnet,dc=cz # A perunResource dn: perunResourceId=123456,perunVoId=12345,dc=perun,dc=cesnet,dc=cz objectclass: top objectclass: perunGroup perunResourceId: 123456 perunFacilityId: 123 assignedGroupId: 753684 == LDAP - entities and attributes description == * PerunPerson or PerunUser - for user image in LDAP (taken from Perun) perunId - main user ID in form of number mail - user email taken from his profile, not necessary preffered one preferredMail - preffered email entryStatus - not ised at the moment, user is active in all cases or does not exist in LDAP sn - user's surname givenName - user's name cn - name and surname (without middleName) o - user's organisation uidNumber;x-ns-* - user's UID in particular namespace (marked as *). For example cesnet, metacentrum etc. login;x-ns-* - user login in particular namespace (marked as *). For example cesnet, metacentrum etc. userCertificateSubject - subject of one user certificate (can be multivalue) eduPersonalPrincipalNames - external logins of all IDP users external identities (can be multivalue) memberOf - defines membership in particular group isServiceUser - being service user value is 1, otherwise value is 0 memberOfPerunVo - defines ID of Vo where user is member (active one) libraryIDs - IDs in library * PerunVo, defines image of VO in LDAP perunVoId - main VO ID o - VO short name description - VO long name uniqueMember - defines relationship between user and the vo (members group) * PerunGroup, image of group under particular VO in LDAP cn - group name perunUniqueGroupName - group name with all parent group and also with Vo name in notation (VoName:parentGroupName:groupName) description - group description perunGroupId - main group ID perunParentGroup - main parent group ID (if exist) assignedToResourceId - resource ID where group is assigned to uniqueMember - defines relationship between user and the group in the direction of "user is member in the group" owner - group owner (if exists) * PerunResource, image of resource under particular VO in LDAP perunResourceId - id of resource in perun perunVoId - id of vo where this resource exists perunFacilityId - id of facility where this resource is assigned to assignedGroupId - ids of assigned groups to this resource = LDAP - entity searching= * Searching by command in CLI: ldapsearch -x -H ldap://localhost:389/ -b "dc=perun,dc=cesnet,dc=cz" "(perunUserId=13889)" - where ldap://localhost:389/ is necessary to replace by server address for remote access - "(perunUserId=13889)" is searching condition (could be changed to perunVoId=, perunGroupId= etc.) - filter can be added at the end, for example ldapsearch -x -H ldap://localhost:389/ -b "dc=perun,dc=cesnet,dc=cz" "(perunUserId=13889)" sn cn givenName (return sn, cn and givenName) = Peruna configuration files= Location: /etc/perunv3 * jdbc.properties - database connection definition jdbc.url=jdbc:oracle:thin:@//svarog.ics.muni.cz:1522/svarog jdbc.username= jdbc.password= jdbc.driver=oracle.jdbc.driver.OracleDriver * perun.properties - many options for Perun run # Perun administrators perun.admins = perunTests, perunController, perunEngine, perunRegistrar, perunSynchronizer, perunCabinet # Grouper TopLevel Stem perun.grouper.toplevelstem = perunMeta # Principals for the Services perun.service.principals = perunv3/engine@META # Principals for the Engines perun.engine.principals = perunv3/engine@META # Principals for the Synchronizer perun.synchronizer.principals = perunSynchronizer # Principals for the Registrar perun.registrar.principals = perunv3-registrar@META # Principals for the Notificator perun.notification.principals = perunNotifications # Principal for the RPC perun.rpc.principal = perunRpc # DB type (oracle/postgresql) perun.db.type = oracle # Default group synchronization interval in fold of 5 minutes perun.group.synchronization.interval = 1 # Timeout for group synchronization in minutes perun.group.synchronization.timeout = 10 # Users who can do delegation perun.rpc.powerusers = perunv3-registrar@META # Perun DB Name perun.perun.db.name = v3dev # Perun RT URL perun.rt.url = https://rt3.cesnet.cz/rt/REST/1.0/ticket/new # Perun service user for RT tickets perun.rt.serviceuser.username = perunv3-rt # Perun service user for RT tickets perun.rt.serviceuser.password = password # Program which ensures password changes perun.passwordManager.program = /usr/local/bin/perun.passwordManager * ldapc.properties ldap.url=ldap://localhost:389 ldap.userDn= ldap.base=dc=perun,dc=cesnet,dc=cz ldap.password= * notif.properties notif.mailSmtpAuth=false notif.username= notif.password= notif.smtpHost=localhost notif.port=25 notif.emailFrom=perunv3@metacentrum.cz notif.fromText=perunv3@metacentrum.cz notif.sendMessages=true notif.starttls=false notif.jabber.jabberServer=jabber.org notif.jabber.port=5222 notif.jabber.serviceName=jabber.org notif.jabber.username= notif.jabber.password= notif.dispatcherName=notifications * perun-extSources.xml - configuration file with external resources. When changed, you must run ./loadExtSourcesDefinitions * log4j.properties - definition goals of logging = Data reinitialization from LDAP to Peruna = * Several steps must be done to get consistent data as result. 1. Get the current highest ID of auditer record (max(ID) from database, highest number from "Audit log" in GUI), for example id-before=250 2. Find initialization script at machine metacentrum-ostra and run it: /home/perunv3/perun-ldapc/inicialize-ldap.sh 3. Wait until script finishes its run and then check highest ID (same way as first step), for example id-after=260. 4. Check whether processes of moving data from Perun to LDAP could not change consistency (in doubt repeat whole process from step on 5. When all is OK, change record "name=ldapConsumer" in table "auditer_consumers", value "last_processed_id". New value is id-before, for example last_processed_id=250. Commit change. 6. Last step is to run LDAP Connector in Hudson in tab meta named "perun-ldapc-deploy-Meta" -> build * Please note: There are two log files of LDAP connector to see its behaviour: var/log/perunv3/ldapc.log var/log/perunv3/perun-ldapc-out.log [[Kategorie:Perun]] [[Kategorie:English]] 4hv5nbfe6cjwczxsg1dqjhz8j8z4knr 0 110 336 335 2016-01-23T15:48:53Z Root 1 naimportována 1 revize wikitext text/x-wiki {{Template:Perun-en-template}} == GUI == You can log in to Perun's graphical web interface using three different authentication methods: using MetaCentrum Kerberos - [https://perun.metacentrum.cz/krb/gui/ Perun GUI] using Federation identity - [https://einfra.cesnet.cz/fed/gui/ Perun GUI] using IGTF Certificate - [https://perun.metacentrum.cz/cert/gui/ Perun GUI] == RPC == RPC is an interface for remote access to Perun using standard HTTP/HTTPS calls. RPC expect you to use GET type of requests for read only methods and POST type of requests for any state changing method (create, update, delete,...). URL structure of each RPC call is following: <nowiki>https://base_url/authz_type/perun_component/data_exchange_format/managerName/methodName?param1=value&param2=value&listParam[]=value1&listParam[]=value2</nowiki> === RPC server location + authz === Following base URLs are provided, each uses different authentication method to log you in. Please note, that if you run your own Perun instance, base URL is different. https://'''perun.metacentrum.cz/krb/rpc/''' (using Kerberos from realm @META) https://'''perun.metacentrum.cz/krb-einfra/rpc/''' (using Kerberos from realm @EINFRA) https://'''perun.metacentrum.cz/fed/rpc/''' (using credentials from your federation IdP) https://'''perun.metacentrum.cz/cert/rpc/''' (using IGTF certificate) === Data exchange formats === Following formats are supported for data exchange: JSON, JSONP and VOOT (soon). You can specify format in URL following example: <nowiki>https://perun.metacentrum.cz/krb/rpc/</nowiki>'''json/''' <nowiki>https://perun.metacentrum.cz/krb/rpc/</nowiki>'''jsonp/''' <nowiki>https://perun.metacentrum.cz/krb/rpc/</nowiki>'''voot/''' (not yet deployed) '''JSONP''' uses padding before and after data to help you match call and it's response. Then, all requests must contain parameter ''callback=matcher'' and returned value is in a format: ''matcher(standard_json_data);''. If callback matcher is not provided, then ''null(standard_json_data);'' is returned instead. If parameter is empty, then ''(standard_json_data);'' is returned. If returned value is not an object, but single value like: string, number or boolean, then ''matcher(value);'' is returned (e.g. ''matcher(false);'' or ''matcher("text");''). === Calling a method === In order to call any method you must provide manager's and method's name in request URL. Both of them use [http://en.wikipedia.org/wiki/CamelCase CamelCase] in their names. [http://perun.metacentrum.cz/javadoc-rpc/ List of available methods] can be found in RPC's javadoc. <nowiki>https://perun.metacentrum.cz/krb/rpc/json/</nowiki>'''vosManager/getVos''' # will give you list of your VOs in JSON format You must use POST for calling state changing methods (create, add, assign, update, delete, remove,...). You can use GET (or POST) for all other methods, which just reads data from Perun (get, find, list,...). Exception is thrown if HTTP request is of different type than method accepts. === Passing parameters to methods === '''GET''' ''Location:'' All parameters must be present in HTTP request URL. Request body is ignored. <br />''Accept:'' string, integer and list (of strings and integers) <br />''Format:'' First parameter in URL starts with ''?'', others with ''&''. List parameters uses ''[]'' after parameter's name and parameters are repeated for every value of list: ''?listParam[]=value1&listParam[]=value2''. <nowiki>https://perun.metacentrum.cz/krb/rpc/jsonp/vosManager/getVoById</nowiki>'''?callback=call1&id=123''' # will give you VO by it's ID = 123 in JSONP format # response like: call1({"name":"vo_name_value","shortName":"vo_short_name_value", "id":123, "beanName":"Vo", ...other vo params... }); '''POST''' ''Location:'' All parameters must be passed in HTTP request body. <br />''Accept:'' any type of correctly defined data / objects <br />''Format:'' defined by data_exchange_format Please note, that '''JSONP also expect JSON data, since padding definition is part of the request URL.''' For JSON/JSONP to pass set of parameters simply create empty JSON object and fill it with properties with names mathing method's parameters and values matching method parameter's types. When passing whole object to method, some of it's properties might be ignored (and not required to be present). E.g. ''beanName'' which is property defining object type. It is contained in objects returned to you, but not required when posting object back to Perun. URL: <nowiki>https://perun.metacentrum.cz/krb/rpc/jsonp/vosManager/addAdmin</nowiki>'''?callback=call2''' DATA: '''{"vo":123, "user":1234}''' # if method requires simple integer parameters with names 'vo' and 'user' like method ''vosManager/addAdmin'' # response like: call2(null); (since method doesn't return anything) === Error handling === When using JSON or JSONP data exchange format, all calls ending with an error are returned as OK (HTTP return code 200). This allow you to process returned exception serialized in expected format (otherwise browsers throw away error responses and your application wouldn't know what's wrong). e.g. callbackPost({"name":"exception_name", "message":"exception_message", "errorId":"1428f68a462" ...other optional parameters... }); == PHP Library == Simple PHP Perun binding. PerunRpcClient.php and examples can be downloaded from GitHub [https://github.com/CESNET/perun/tree/master/perun-php https://github.com/CESNET/perun/tree/master/perun-php]. === Prerequisities === Install PHP support and curl extension for Apache: sudo apt-get install php5-curl libapache2-mod-php5 Setup Apache to run CGI scripts in folder with your app: Options Indexes MultiViews Includes ExecCGI FollowSymLinks AddHandler cgi-script .php === Configuration === Edit PerunRpcClient.php and adjust next options: // PERUN RPC URL const RPC_URL = "https://perun.cesnet.cz/krb/rpc/json/"; // Choose the authentication method: // Username const USER = "perun-client"; // Password based authentication, left empty if you are using Kerberos authz const PASSSWORD = ""; // Kerberos based authentication, left empty if there is no Kerberos support const KERBEROS_CC = "/tmp/krb5cc_perun-client"; For local development you can set USER to "your_login@REALM" and PASSWORD as your kerberos password. Then keep KERBEROS_CC empty. This is not secured solution !! === Calling a method === In order to call any method you must provide manager's and method's name. Both of them use [http://en.wikipedia.org/wiki/CamelCase CamelCase]. [http://perun.metacentrum.cz/javadoc-rpc/ List of available methods] can be found in RPC's javadoc. include ("PerunRpcClient.php"); Initialize the Perun Client $client = new PerunRpcClient(); Method without parameters $vos = $client->retrieveData("vosManager/getVos"); Method with parameters $params = array('vo' => 21); $members = $client -> retrieveData("membersManager/getRichMembers", $params); If something failed, retrieveData() will return false. if($members === false){ print "Error while getting response."; exit; } Process the results foreach($vos as $vo){ print {$vo->id}; print {$vo->shortName}; print {$vo->name}; } foreach($members as $richMember){ print {$richMember->user->firstName}; print {$richMember->user->lastName}; } == JavaScript Library == For using you will need configuration.js, where will be specificated rpc_url. More about rpc_url [https://wiki.metacentrum.cz/wiki/Perun_remote_interfaces#Data_exchange_formats here]. Configuration = { RPC_URL : "https://perun.metacentrum.cz/fed/rpc/jsonp/", } In order to call any method you must provide manager's and method's name to url. Both of them use [http://en.wikipedia.org/wiki/CamelCase CamelCase]. [http://perun.metacentrum.cz/javadoc-rpc/ List of available methods] can be found in RPC's javadoc. $.ajax({ url : Configuration.RPC_URL + "groupsManager/getGroupRichMembersWithAttributes", data : { group : 817 }, success : function(data, textStatus, jqXHR) { if(typeof data.errorId !== "undefined") { alert(data.errorText); }else{ //definition of table var table = PerunTable.create(); table.addColumn("user.id", "ID"); table.addColumn("user.firstName", "First name"); table.addColumn("user.lastName", "Last name"); // adds the values to the table table.add(data); //create html from table var tableHtml = table.draw(); //add table to specific div element $("specificDiv").html(tableHtml); } }, dataType: "jsonp", type : "get" }) Class PerunTable is easy way to visualize accepted data. == Synchronization a group with external resource == SQL query returning column '''firstName''', '''lastName''', '''login'''. Optionally ''middleName'', ''titleBefore'', ''titleAfter''. SQL query could return columns with names of concrete attributes. Due to size restriction, attribute names must be shorten, where x:y:z means: {| style="border: solid 1px grey;" cellpadding="5" cellspacing="0" | style="font-weight: bold; background-color: lightgrey;" | x || style="font-weight: bold; background-color: lightgrey;" | meaning |- |m || member |- |u || user |- |f || facility |- |r || resource |- |g || group |- |v || vo |- |h || host |- |mr || member_resource |- |uf || user_facility |- |gr || group_resource |} {| style="border: solid 1px grey;" cellpadding="5" cellspacing="0" |style="font-weight: bold; background-color: lightgrey;" | y || style="font-weight: bold; background-color: lightgrey;" | meaning |- |d || def |- |o || opt |} Examples: m:d:address -> urn:perun:member:attribute-def:def:address u:d:login-namespace:mu -> urn:perun:user:attribute-def:def:login-namespace:mu * Example groupMembersQuery for external resource SQL database ''cz.metacentrum.perun.core.impl.ExtSourceSql'' select name as first_name, surname as last_name, login, mail as "m:d:mail", telephone as "m:d:phone" from someTable * Example groupMembersQuery for external resource LDAP ''cz.metacentrum.perun.core.impl.ExtSourceLdap'' TBA * Example groupMembersQuery for external resource ISMU ''cz.metacentrum.perun.core.impl.ExtSourceISMU'' https://is.muni.cz/auth/export/skupina_osob.pl?format=csv&kodovani=utf-8&skup_zkratka=1433:15 How to create a synchronized with external resource PERUNPEOPLE type SQL with five minute interval: ./setGroupAttribute --groupId 123 --attributeName 'urn:perun:group:attribute-def:def:groupExtSource' --attributeValue PERUNPEOPLE ./setGroupAttribute --groupId 123 --attributeName 'urn:perun:group:attribute-def:def:groupMembersQuery' --attributeValue 'select name as first_name, surname as last_name, login, mail as urn:perun:member:attribute-def:def:mail, telephone as urn:perun:member:attribute-def:def:phone from someTable' ./setGroupAttribute --groupId 123 --attributeName 'urn:perun:group:attribute-def:def:synchronizationEnabled' --attributeValue true ./setGroupAttribute --groupId 123 --attributeName 'urn:perun:group:attribute-def:def:synchronizationInterval' --attributeValue 1 Manual synchronization ./synchronizeGroup --groupId 123 [[Category:Perun]] [[Category:For managers]] [[Category:English]] ia3z5jc9ciwsnfu0q399igu8ihvqrbv 0 111 338 337 2016-01-23T15:48:54Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Use cases|Back to Use cases]] [[Less frequent tasks|Back to Less frequent tasks]] === Assign tag to resource === [[My role is VO MANAGER|''VO MANAGER]] ''role needed'' * visit {{PerunGUILink|rel=vo/list perunGUI VOs pages}} * use '''Select VO''' [[File:Building.png|frameless|alt=VO|]] from the left menu to display list of VOs which you can manage * click on the row with VO name to select VO which you want to work with * click on the button '''Resources''' at upper bar OR click on the '''Resources''' [[File:server_group.png|frameless|alt=VO|]] option at the left menu * select resource by clicking on the row with name of required resource * click on the button '''Tags''' and list of recently assigned tags is displayed * press the button [[File:add.png|frameless|alt=VO|]] '''Add...''' and list of tags for adding is offered * select required tag (or tags) by ticking check-box, click the button '''Add''' [[File:add.png|frameless|alt=VO|]] * required tag is assigned to this resource * click on button [[File:stop.png|frameless|alt=VO|]] '''Cancel''' if you want to cancel adding of tags [[Category:Tutorials]] [[Category:Perun]] [[Category:English]] arz04xy2amoc01oncu8qc1db45o79g7 0 112 340 339 2016-01-23T15:48:54Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Use cases|Back to Use cases]].....OR.....[[Less frequent tasks|Back to Less frequent tasks]] == Create resource tags == To resouces can be assigned [[Conceptual_scheme_and_definition_of_terms#Purpose_of_resource_-_Resource_tags|tags]] which marked special properties of resources. [[My role is VO MANAGER|''VO MANAGER]] ''role needed'' * visit perunGUI VOs [[File:Building.png|frameless|alt=VO|]] pages in {{PerunGUILink|rel=vo/list GUI}} * use '''Select VO''' from the left menu to display a list of VOs which you can manage * click on the row with a VO name where you want to create the tag * click on the '''Resource tags''' at left menu [[File:tag.png|frameless|alt=VO|]] option and list of recently created tags is displayed * use '''Create...''' [[File:add.png|frameless|alt=VO|]] button * fill ''Tag name'' field and press '''Create''' [[File:add.png|frameless|alt=VO|]] button in small frame * use [[File:stop.png|frameless|alt=VO|]] '''Cancel''' to cancel creating of new tag * new tag is listed at ''VO resource tags'' page * this newly created tag can be assigned to resources as explained at [[Assign tag to resource]] [[Kategorie:Perun]] [[Kategorie:Tutorials]] [[Kategorie:English]] 69e3sv05yl7xhs0irfiwvq1a7p1b20k 0 113 342 341 2016-01-23T15:48:54Z Root 1 naimportována 1 revize wikitext text/x-wiki {{Perun-en-template}} ==My profile== ====User overview==== Section where all your personal information are stored. You can edit it with an exception of fields ''Organization'' and ''Email''. These two are taken from external identity provider; therefore they can not be changed. Please, keep all your information up-to-date, because this contact (preferably email or phone number) would be used by our user support to contact you in case of need. The very important feature is possibility to '''change a password''' in the section ''Authentication''. Your '''public SSH key''' can be saved here also. There is a possibility to see all your identities and particular identity providers in section '''External identities'''. To add new identity, use component identity consolidator. ====VO tabs==== Each tab covers user's membership status and his membership in groups in a particular VO. By clicking on ''Quota and shell for resources'', you can '''request for higher disk quota''' in selected VO's resource or''' switch your default shell '''in VO's resource. When you feel lost in VO's resources, try to ask your VO manager. ==My publications== MetaCentrum users can add a publication with acknowledgement to VO MetaCentrum, based on them, user can gain particular benefits. This feature serves '''for VO MetaCentrum users only'''. Other VO's do not support this feature so far, so there is no point in adding publications by non MetaCentrum members. ==My applications== List of user's application forms and their states used for enrollment to VOs. By clicking on each record, detailed information is obtained. The information consists of [[Create_application_form#Application_types|application type]] and [[Registration_as_member_of_VO#Application_workflow|application state]]. This section serves as an evidence only, to apply for a VO or group membership, please use particular [http://perun.cesnet.cz/web/guide.php application forms]. ==Service identities== [[Conceptual_scheme_and_definition_of_terms#An_account_shared_among_users_--_Service_member|Service member]] serves primary for service purposes. This account can be shared among one or more members. Creating service identity is described [[Create_service_member|here]]. Most of users will never obtain service identity, but when you get some, you can see its detail in this section. By clicking on line with your service identity, you get more detailed information. [[Category:Perun]] [[Category:For managers]] [[Category:English]] nt348xwiv135yh3320h7rcvjfwxkg3u 0 114 344 343 2016-01-23T15:48:54Z Root 1 naimportována 1 revize wikitext text/x-wiki {{Template:Perun-en-template}} System Perun is designed as a composition of mutually cooperating modules. The modularity is considered as a great advantage because any subset of modules containing Core will work as an independent system. [[File:Structure_overall_new.png|center|100px|frameless|700px|alt=Component_schema.|Component_schema.]] ==Auditer== Component creating and logging all events occurred in system. This component can push events of predefined type to any other component. ==Cabinet== This component is serves for MetaCentrum purposes only where records containing publications with acknowledgment to MetaCentrum are stored. Based on them, users will obtain particular benefits. ==Controller== Component communicating via database with Engine and Dispacher and cooperating with Engine and Dispatcher on propagation management. ==Core== Core of Perun system. ==Dispatcher== Internal events are parsed in this component. When event related with any service is detected, it is send to Engine. ==Engine== Engine processes events and propagates new state to affected destinations. First, it calls GEN scripts to prepare new configuration files. Consequently, SEND scripts send configuration files generated in first step to predefined destinations (to email address, to http web page, to machine via SSH). As an advantage, it works with a tiny delay to send events in a batch. The aggregation of events serves to smoother propagation via system. SLAVE script is stored in each particular machine. It processes configuration file in the machine, does the change (e.g. update new line in passwd) in resource and as a response returns code to SEND script, which report it back to Perun system. Moreover, hooks to program new features into SLAVE scripts are available. ==Identity consolidator== Web application for user identities consolidation. User can link his account with his identities provided by identity providers. Widely used authentication methods are personal certificates and federation, but any database storing combination "login and password", for example LDAP or Kerberos, can serve as a source of identities for user access to the system. ==LDAPc== LDAP connector. It analyses system events, seeks for particular changes concerning user membership in groups and saves them in LDAP database in nearly real time speed. ==Notifications== This highly customizable component intercepts system log. Based on events found in log, it does specific predefined action, for example send an email to particular address, aggregate events or send message after specific time period. Whole system is fully functional but still in a process of development. ==Registrar== This component maintains whole process of enrollment, not only creating and customizing application form but also customizable email notifications and regular account extension. ==RPC== Interface connecting several components (see first figure) including Core. It uses his own protocol (based on HTTP and JSON) or VOOT protocol as an alternative . [[Category:Perun]] [[Category:English]] 9ckfaywfecxm26v5fyfanri5ercwi4x 0 115 346 345 2016-01-23T15:48:54Z Root 1 naimportována 1 revize wikitext text/x-wiki {{Template:Perun-en-template}} Just an overview how all the components and entities are interconnected in Perun. [[File:Perun_Schema.png|center|100px|frameless|700px|alt=Perun_schema in detail.|Perun_schema.]] [[Category:Perun]] [[Category:English]] fr5iv8ehypgdepnyxzrnj8wxvx7vyhy 0 116 348 347 2016-01-23T15:48:55Z Root 1 naimportována 1 revize wikitext text/x-wiki {{Template:Perun-en-template}} Schema showing how user can get into the Perun, how is organized and finally how information about his/her access rights are propagated to the end services. [[File:PerunEnrollment.png|center|100px|frameless|400px|alt=Perun enrollment schema.|Perun_enrollment_schema.]] [[Category:Perun]] [[Category:English]] fe27dtozxc9l5r2usjhhy0m2633rmqe 0 117 350 349 2016-01-23T15:48:55Z Root 1 naimportována 1 revize wikitext text/x-wiki {{Template:Perun-en-template}} ==Introduction== Speaking about propagation, we mean distribution of changes made in system to the particular destinations. For example, simple action as adding a new VO member by VO manager has large consequences as creating /home directories, setting an access rights, adding line in passwd... etc. for the user at all VO's resources Propagation does not only transport completely new state to destinations, but also runs pre-installed scripts on machines to make all necessary changes (configure services, create directories, set access rights...). Propagation of new state has several significant advantages. Sometimes happened situation when destination must be reinstalled or new hardware is added into infrastructure. Its inner state is lost or empty, but only one update from Perun, gives it new up-to-date state. Consequently, destinations keep the state until the new one is send by Perun. Destination can run on their own also when Perun system would not work. Package of open source bash scripts to run on destinations is delivered with Perun installation. There are fully functional and ready to run, but when administrator wants, he could edit or customize them, or even more, replace with his own scripts. He has the control about what Perun does in destination fully in his hand. ==Propagation workflow== [[File:perun_propagace.png|center|100px|frameless|800px|alt=Perun propagation schema.|Perun_propagation.]] Considering action when VO manager has added a new member into his VO, information about this change goes to Audit log. This log is read by Engine, so a change influencing resources is detected and processed. First, engine calls GEN scripts. These GEN scripts generate complete new state of the destination including the current change (e.g. new line in passwd). As a second step, engine calls SEND script. This script takes new state generated in first step by GEN script and sends it to destination via SSH (most common way) or any other channel (for example email or jabber ). As a third step, SEND script triggers SLAVE script "perun" stored in destination. Perun usually uses root access to run SLAVE script in a destination, but often Perun simply does not need it. It can run as any user, but the user has to have a proper rights to run all associated SLAVE scripts. SLAVE script identifies type of service and checks versions of service script. This controll is important due to new versions of SLAVE scripts. When version of service script matches with version of SLAVE script, then, predefined SLAVE script configuring particular service in destination is run. Sometimes destinations have specific requirements (e.g. settings of home directories, paths to scripts etc.). To fullfil this requirements without change of slave script, pre and post scripts are used. They can complement functionality of slave scripts. When slave script starts, it searches for pre_ script in the service directory and runs it before its own functionality. The same process is after slave script execution, the post_ scripts are run. Use of script is optional, but it can be utilized by administrator, who want to add a new functionality to our Perun scripts. Pre and post scripts simply bring higher level of customization. After finishing the whole process, as a response SLAVE script returns status to SEND script, which report it back to Perun system. All return codes obtained from destination are checked on regular basis (e.g. once per hour) and when error code has been occured, repropagation can be triggered. ==Propagation and security== As you already know, service (for example ssh, ftp, web server) is assigned to destination in system Perun by Facility manager. Destination is a target, where configuration of service is propagated. There are several types of destination, for example email (new configuration is send to the given email address), but in your case we use type HOST (new configuration is send directly to the machine). When destination accepts the configuration, pre-installed scripts are run on destination to make all necessary changes. For better insight, see figure [[Propagation#Propagation_workflow|Propagation workflow]]. Perun does not send only configuration from facility to machine, but this configuration also contains data about "destination" (hostname of machine receiving configuration from Perun) and "facility" (facility for which configuration is built). These attributes are further important for machine to distinguish whether accept configuration or not. Facility manager (for example manager of facility ''carach.ics.muni.cz'' serving as cloud frontend) can assign as destination machine's name (for example ''carach1.ics.muni.cz'') or alias (for example ''cloud-frontend.ics.muni.cz'') or its IP address in Perun. The last two possibilities (alias, IP) could cause problems because manager can insert an alias or IP out of scope of his rights. As example, evil facility manager of ''skirit.ics.muni.cz'' can assign as destination alias ''cloud-frontend.ics.muni.cz'' as a destination for some service, despite the fact that he is not manager of machine with alias ''cloud-frontend.ics.muni.cz''. The machine is registered in the Perun, but its alias is not present. Because evil facility manager assigned as destination alias, which is not in the system, inner control mechanism can not avert this attempt to fraud behavior. The inner control mechanism will be further described later on in the text. Since evil facility manager saved this settings, machine ''carach.ics.muni.cz''. with alias ''cloud-frontend.ics.muni.cz'' will receive configuration from facility ''skirit.ics.muni.cz'' which is under full control of the evil manager. Perun has inner system of control whether facility manager has right to assign service to destination. It covers situation when assigned value is stored in Perun database. Inner control mechanism consists of following rules: The facility manager can add name of new destination (or host) to service if and only if * There is no host and no destination with the name in the system. * There is a destination or host connected with the facility, where the manager is facility manager. Inner control mechanism does not cover situation when facility manager assign value of destination out of Perun coverage (for example new alias of destination). To avoid potentially fraudulent behavior, destination accepts configuration where '''value of attribute "destination" equals to the result of command `hostname -f` as default'''. To change default settings, destination (host) manager must fill variables DNS_ALIAS_WHITELIST or FACILITY_WHITELIST with values (IPs, aliases, destination's name). These values are then compared with attributes "destination" and "facility" delivered with new configuration. These variables are set in: '''/opt/perun/bin/perun''' or in '''/etc/perunv3.conf''': # accept configuration only if it was send to one of these hostnames # prevent someone to configure perun to send malicious configuration via dns alias or ip address DNS_ALIAS_WHITELIST=( `hostname -f` ) FACILITY_WHITELIST=() # from which facilities this host accepts configuration, # insert facility name as is in Perun. DNS_ALIAS_WHITELIST accepts both IP addresses and aliases (only one value is enough for one machine), values are separated by space. For example DNS_ALIAS_WHITELIST = (86.49.81.84 cloud-frontend.ics.muni.cz) FACILITY_WHITELIST accepts facility names (as are named in Perun), values are separated by space. For example FACILITY_WHITELIST=(carach.ics.muni.cz) To allow service propagation, both whitelists are checked and match in only one whitelist is enough to accept the configuration. [[File:perun_bezpecnost.png|center|100px|frameless|800px|alt=Perun propagation security schema|perun_bezpecnost]] [[Category:Perun]] [[Category:English]] 8rgs7t3bdfidvh4enqcz3ateorsh69e 0 118 352 351 2016-01-23T15:48:55Z Root 1 naimportována 1 revize wikitext text/x-wiki {{Template:Perun-en-template}} == Concept of namespaces == Concept of namespaces allows users to have different lognames at various entities of system. Each facility is assigned to specific namespace. User who want to use facilities from different namespaces have to own lognames from all these namespaces of facilities. This rule bears on every date related to loging on facilities (lognames, UIDs, GIDs,...). All data and options related to namespases are kept at following attributes: * For '''facility''': ** '''<nowiki>urn:perun:facility:attribute-def:def:login-namespace</nowiki>''' defines namespace for all user's logins on facility. ** '''<nowiki>urn:perun:facility:attribute-def:def:uid-namespace</nowiki>''' defines namespace for user's UIDs on facility. ** '''<nowiki>urn:perun:facility:attribute-def:def:unixGID-namespace</nowiki>''' defines namespace for unix groups GIDs on facility. ** '''<nowiki>urn:perun:facility:attribute-def:virt:maxGID</nowiki>''' maximal value for GID on facility - it is determined automatically based on facility's namespace. ** '''<nowiki>urn:perun:facility:attribute-def:virt:minGID</nowiki>''' minimal value for GID on facility - it is determined automatically based on facility's namespace. ** '''<nowiki>urn:perun:facility:attribute-def:virt:maxUID</nowiki>''' maximal value for UID on facility - it is determined automatically based on facility's namespace. ** '''<nowiki>urn:perun:facility:attribute-def:virt:minUID</nowiki>''' minimal value for UID on facility - it is determined automatically based on facility's namespace. ** '''<nowiki>urn:perun:facility:attribute-def:def:unixGroupName-namespace</nowiki>''' defines namespace for unix groups names on facility. * For '''groups''': ** '''<nowiki>urn:perun:group:attribute-def:def:unixGID-namespace:</nowiki>''name_of_namespace''''' defines unix GID in namespace ''name_of_namespace'' for main group on facility. ** '''<nowiki>urn:perun:group:attribute-def:def:unixGroupName-namespace:</nowiki>''name_of_namespace''''' defines unix group name in namespace ''name_of_namespace'' for main group on facility. * For '''resources''': ** '''<nowiki>urn:perun:resource:attribute-def:def:unixGID-namespace:</nowiki>''name_of_namespace''''' defines unix GID of group which represented group for whole resource at namespace ''name_of_namespace''. ** '''<nowiki>urn:perun:resource:attribute-def:def:unixGroupName-namespace:</nowiki>''name_of_namespace''''' defines unix group name of group which represented group for whole resource at namespace ''name_of_namespace''. * For '''users''': ** '''<nowiki>urn:perun:user:attribute-def:def:login-namespace:</nowiki>''name_of_namespace''''' defines users's logname in namespace ''name_of_namespace''. ** '''<nowiki>urn:perun:user:attribute-def:def:uid-namespace:</nowiki>''name_of_namespace''''' defines users's UID in namespace ''name_of_namespace''. ** '''<nowiki>urn:perun:user:attribute-def:def:preferredUnixGroupName-namespace:</nowiki>''name_of_namespace''''' defines list of user's preferred unix group names, ordered by user's personal preferrences for ''name_of_namespace'' namespace. * There are several attributes which are not related to any entity. They are defined for each namespace, used on facility. They are target for calculation of appropriate facility virtual attributes (mentioned above): ** '''<nowiki>urn:perun:entityless:attribute-def:def:namespace-maxGID</nowiki>''' contains basic value of maximal GID. ** '''<nowiki>urn:perun:entityless:attribute-def:def:namespace-minGID</nowiki>''' contains basic value of minimal GID. ** '''<nowiki>urn:perun:entityless:attribute-def:def:namespace-maxUID</nowiki>''' contains basic value of maximal UID ** '''<nowiki>urn:perun:entityless:attribute-def:def:namespace-minUID</nowiki>''' contains basic value of minimal UID [[Category:Perun]] [[Category:English]] 0mgwukloldjmwonug6xttaepj9otw1n 0 119 354 353 2016-01-23T15:48:56Z Root 1 naimportována 1 revize wikitext text/x-wiki __NOTOC__ ==Správa uživatelů v sytému Perun== Perun je systém pro správu účtů, kde administrátoři Virtuální Organizace (VO) řídí přístupy (schvalují nebo zamítají uživatele), agregují uživatele do skupin, vytvářejí servisní uživatele (účty pro skripty/programy/systémy). Více o systému Perun na [http://perun.metacentrum.cz/web/index.shtml webu aplikace]. '''Odkazy pro přihlášení:''' Vyberte způsob přihlášení, který je pro vás nejpohodlnější. Pokud nevíte, zvolte federaci eduID.cz. [https://einfra.cesnet.cz/fed/gui/ Přihlášení pomocí účtu ve federaci eduID.cz.] [https://einfra.cesnet.cz/krb/gui/ Přihlášení pomocí Kerberos identity.] [https://einfra.cesnet.cz/cert/gui/ Přihlášení pomocí osobního certifikátu.] === Návody === * [[Zasilani pozvanek do VO|Zasílání pozvánek do VO]] * [[Schvalovani uzivatelu|Schvalování uživatelů]] * [[Mazani uzivatelu|Mazání uživatelů]] * [[Prodlouzeni clenstvi ve VO|Prodloužení členství ve VO]] * [[Vytvoreni servisniho uctu|Vytvoření servisního účtu]] * [[Pridani uzivatelu k existujicimu servisnimu uctu|Přidání uživatelů k existujícímu servisnímu účtu]] * [[Zmena hesla servisniho uctu|Změna hesla servisního účtu]] * [[Zalozeni skupiny|Založení skupiny]] * [[Nastaveni kvot na ulozisti|Nastavení kvót na úložišti]] <!--=== Tutorials === Lost in Perun system? Our step-by-step tutorials help you. * [[Use cases|Tutorials and use cases]] === Advanced tutorials === Do you want dig deep in Perun system? Interesting facts for you. * [[Registration as member of VO| How to create an application form - long version]] * [[Perun CLI]] * [[What I can do as 'ROLE']] * [[Perun remote interfaces|Perun remote interfaces]] * [[Perun external sources|Perun external sources]] * [[Synchronization of groups with external source|Synchronization of groups with external source]] === The Perun project === Do you really want to know how Perun works? * [[Conceptual scheme and definition of terms]] * [[Perun overall schema]] * [[Perun component schema]] * [[Perun enrollment schema]] * [[Propagation]] * [[Namespaces]] === Technical documentation === * [[Known bugs]] {{info|If your issue is not covered by our manual, please report your problem to [mailto:perun@cesnet.cz perun@cesnet.cz]}} --> [[Kategorie:Pro správce]] [[Kategorie:Datové úložiště]] [[Kategorie:Česky]] [[Kategorie:Perun]] h25482pi8c39s8fhbu60rocy35cfqiy 0 120 356 355 2016-01-23T15:48:56Z Root 1 naimportována 1 revize wikitext text/x-wiki {{Template:Perun-cs-template}} [[Conceptual_scheme_and_definition_of_terms#Virtual_organization_-_VOs|VO]] manager is a person or group of people allowed to manipulate with VO data. First manager of particular VO must be created by Perun admin.Vo manager can utilize both CLI interface and {{PerunGUILink|rel= GUI web GUI}} to operate system entities. VO manager is responsible for following tasks: == [[zasilani_pozvanek_do_vo|Zasílání pozvánek do VO]] == == [[schvalovani_uzivatelu|Schvalování uživatelů]] == == [[mazani_uzivatelu|Mazání uživatelů]] == == [[prodlouzeni_clenstvi_ve_vo|Prodloužení členství ve VO]] == == [[vytvoreni_servisniho_uctu|Vytvoření servisního účtu]] == == [[zmena_hesla_servisniho_uctu|Změna hesla servisního účtu]] == == [[zalozeni_skupiny|Založení skupiny]] == == [[nastaveni_kvot_na_ulozisti|Nastavení kvót na úložišti]] == == Adding users among [[Conceptual_scheme_and_definition_of_terms#Users_into_VO_-_Members|VO members]] == VO manager is allowed to select user that will become a member of the VO from [[Conceptual_scheme_and_definition_of_terms#External_identity_resources_-_Ext_sources|external identity resources]] or resources available to the VO. Choosing your VO from the {{PerunGUILink|rel=vo/list VO list}} in GUI, you will see not only details about VO but also menu items in left part. VO manager uses menu to administer VO. Button ''Members'' [[File:user_green.png|frameless|alt=VO|]] serves to list all VO members. Button ''Add'' enables to select new user from list of candidates. Button ''Search'' is used to search for the user. User detail is shown by clicking the user in the list of members. Button ''Remove'' serves to remove member from VO. == Creating a VO manager from VO member == VO manager is allowed to create a VO manager from any other VO member. Detailed tutorial [[Add_VO_manager|how to add VO manager]]. == [[Conceptual_scheme_and_definition_of_terms#Groups_-_Groups|Group]] administration == VO inner structure can be enhanced by hierarchy of groups. Every group has an access rights to particular resources in VO. Group administration is described in a detail in [[Group_administrator%27s_manual#Groups_administration|Group manager's manual]]. == Creating group manager from VO member == VO manager can create a group manager from VO member. How to create a Group manager is described in [[Group_administrator%27s_manual#Creating group manager from VO member|Group manager's manual]]. == Adding VO members into group == VO manager and Group manager can add a VO member into group. Whole process is described in the [[Group_administrator%27s_manual#Adding VO members into group|Group manager's manual]]. == Creating resource tags == VO manager can create tags for resources. Detailed tutorial: [[Create resource tags|how to create resource tag]]. ==[[Conceptual_scheme_and_definition_of_terms#Resources_-_Resources|Resource]] utilization == Resource is part of Facility that can VO utilize. It is created by Facility manager, but only VO manager is allowed to manipulate with it. First, select particular VO in {{PerunGUILink|rel=vo/list GUI}}. Button ''Resources'' [[File:server_group.png|frameless|alt=resource|]] shows list of resources in VO. Button ''Remove'' removes selected Resources. By clicking the row with name of Resource, the detailed information appears. By clicking the tab ''Assigned groups'', the list of assigned groups is listed. Tab ''Assigned services'' shows list of assigned services. By clicking on the button ''Service settings'', page with attributes' settings is shown. These attributes are necessary to proper functionality of services. How to: * [[Add member to resource]] * [[Remove member from resource]] * [[Assign group to resource]] * [[Assign service to resource]] * [[Assign tag to resource]] == Managing an application form == [[Create_application_form|Tutorial to create basic application form.]] [[Approve application to VO|Approve application to VO.]] == Creating rules to account extensions == It is necessary to set attribute ''membershipExpirationRules'' for VO. Attribute can be added in ''Settings'' in VO. Its items can be: * '''doNotAllowLoa''' - list of [[Conceptual_scheme_and_definition_of_terms#Level_of_assurance|LoAs]] separated by comma, which won't be allowed in VO (users can't become members). * '''period''' - time period to extend membership. It can be set as fixed date (without year), e.g. 1.1 or as number of days/months/years with prefix "+" that defines time period that extends membership. Units are d = day, m = month, y = year, e.g. +128d extends account to 128 days. +6m, + 1y. * '''doNotExtendLoa''' - list of [[Conceptual_scheme_and_definition_of_terms#Level_of_assurance|LoAs]] separated by comma, that are not extensible. * '''gracePeriod''' - when present date of initial application or extending request equels extension date minus ''gracePeriod'' then user account is extended to the next time period (''period'' date in next year). Value is in format number days/months/years. Units are d = day, m = month, y = year, e.g. 128d, 6m, 1y * '''periodLoa''' - an exception in period for given LoA. Format of value is: LoA|period[.]. ''LoA'' is given Loa number and ''period'' is in same format as a ''period''. Optional dot at the end means whether extend an account to user with filled ''membershipExpiration'' or not. If dot is present, user with filled membershipExpiration is not allowed to extend an account. == More tasks == [[My_role_is_VO_MANAGER|Full list of all VO manager tasks]] [[Category:Perun]] [[Category:For managers]] [[Category:English]] kboxduewfrggqu7k9b8r5zgf8etilpv 0 121 358 357 2016-01-23T15:48:56Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Perun system - Sprava VO|Zpět na Perun system - Správa VO]] ---- {{warning| Nepoužívejte prosím funkce poskytované systémem Perun, které nejsou popsané v tomto návodu. Mohlo by dojít k narušení struktury Vámi spravovaných VO. Připravujeme přehlednější uživatelské rozhraní. }} <!--[[My role is VO MANAGER|''VO MANAGER]] ''role needed''--> * Přihlaste se do prostředí [https://einfra.cesnet.cz/fed/gui/ Perun] a přejděte do sekce '''VO Manager''' [[File:Building.png|frameless|alt=VO|link=|]]. [[Soubor:Perun_gui.png|link=|]] * V levém menu vyberte položku '''Select VO''', čímž zobrazíte všechny VO, které máte ve správě. * Ze seznamu vyberte požadovanou VO, do které chcete pozvat nové uživatele, otevře se nový panel s nastavením dané VO. * Na záložce '''Overview''' klikněte na tlačítko '''Invite user''' [[Soubor:Invite_user.png|frameless|alt=invite_user|link=|]]. [[Soubor:Perun_vo.png|link=|]] * Do zobrazeného formuláře vyplňte '''Email''' případně volitelně vyplňte i jméno do kolonky '''Name''' a zvolte jazyk zaslané přihlášky v položce '''Language'''. [[Soubor:Perun_invitation_to_vo.png|link=|]] * Formulář odešlete stisknutím tlačítka '''Send invitation''' [[Soubor:Email.png|frameless|alt=Send invitation|link=|]]. * Uživateli přijde na email v následujícím formátu: <pre> Vážená uživatelko, Vážený uživateli, správce Virtuální Organizace (VO) xyz DU Vás zve k využívání datového prostoru této VO. Abyste se stal členem VO, je nutné vyplnit přihlášku.&#10;Pro přístup k přihlášce se musíte autentizovat pomocí federace nebo certifikátu: FEDERATION: https://einfra.cesnet.cz/fed/registrar/?vo=VO_xyz CERTIFICATE: https://einfra.cesnet.cz/cert/registrar/?vo=VO_xyz Stav své přihlášky můžete sledovat v části "Applications" na adrese: https://einfra.cesnet.cz/fed/registrar/?vo=VO_xyz&page=apps O jejím schválení budete informován(a) dalším mailem. -------------------------------------------------- Administrátor VO_xyz du-support@cesnet.cz http://du.cesnet.cz </pre> [[Kategorie:Pro správce]] [[Kategorie:Datové úložiště]] [[Kategorie:Česky]] [[Kategorie:Perun]] 3x1v3u8kfdlznm2so64o1u03lseig2b 0 122 360 359 2016-01-23T15:48:56Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Perun system - Sprava VO|Zpět na Perun system - Správa VO]] ---- <!--[[My role is VO MANAGER|''VO MANAGER]] ''role needed''--> * Pokud uživatel podá příhlášku do VO, kterou spravujete, pak Vám přijde email v obdobném formátu jako je uveden níže. V emailu najdete odkazy směřující na stránku s podrobnými informacemi o uživateli. Na odkazované stránce také máte možnost '''Schválit/Zamítnout''' přihlášku. Vyberte si preferovaný způsob přihlášení, přejděte na odpovídající odkaz z emailu a přihlaste se tak do systému Perun pro zobrazení detailu přihlášky. <pre> Vážený administrátore, byla podána nová přihláška do "Název instituce" pod ID=46456457 uživatelem: "Jméno uživatele", "Email uživatele" / &#10;https://whoami.cesnet.cz/idp/shibboleth. Detail přihlášky s možností přijetí/zamítnutí: FEDERACE: https://einfra.cesnet.cz/perun-gui/#vo/appdetail?id=46456457 KERBEROS (@META): https://einfra.cesnet.cz/perun-gui-krb/#vo/appdetail?id=46456457 CERTIFIKÁT: https://einfra.cesnet.cz/perun-gui-cert/#vo/appdetail?id=46456457 Pokud při vytvoření přihlášky došlo k chybám, výpis následuje: -------------------------------------------------- Váš NGI Perun management team </pre> * Po kontrole přihlášky stiskněte tlačítko '''Approve''' [[Soubor:User_green.png|frameless|alt=Schválit|link=|]] pro schválení přihlášky nebo tlačítko '''Reject''' [[Soubor:Reject.png|frameless|alt=Schválit|link=|]] pro zamítnutí přihlášky. [[Kategorie:Pro správce]] [[Kategorie:Datové úložiště]] [[Kategorie:Česky]] [[Kategorie:Perun]] 867vcmtj0eir70qc1xzrum1er2qg7tw 0 123 362 361 2016-01-23T15:48:57Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Perun system - Sprava VO|Zpět na Perun system - Správa VO]] ---- <!--[[My role is VO MANAGER|''VO MANAGER]] ''role needed''--> * Přihlaste se do prostředí [https://einfra.cesnet.cz/fed/gui/ Perun] a přejděte do sekce '''VO Manager''' [[File:Building.png|frameless|alt=VO|link=|]]. * V levém menu vyberte položku '''Select VO''', čímž zobrazíte všechny VO, které máte ve správě. * Ze seznamu vyberte požadovanou VO, ve které chcete smazat uživatele, otevře se nový panel s nastavením dané VO. [[Soubor:Perun_gui.png|link=|]] * Přejděte na záložku '''Members''' nebo vyberte z levého menu položku '''Members''' [[Soubor:User_green.png|frameless|alt=Members|link=|]]. [[Soubor:Perun_vo.png|link=|]] * Do vyhledávacího pole zadejte počáteční písmena jména nebo login a stiskněte tlačítko '''Search''' [[Soubor:Find.png|frameless|alt=List all|link=|]] nebo stiskněte tlačítko '''List all''' [[Soubor:User_green.png|frameless|alt=List all|link=|]] pro zobrazení všech členů VO. [[Soubor:Perun_members.png|link=|]] [[Soubor:Perun_vyhledani_uzivatele.png|link=|]] * Po nalezení uživatele jej označte pomocí zaškrtávacího pole [[Soubor:Checkbox.png|frameless|alt=List all|link=|]]. [[Soubor:Perun_vyber_uzivatele.png|link=|]] * Stiskněte tlačítko '''Remove''' [[Soubor:Remove.png|frameless|alt=List all|link=|]] a ve vyskakovacím dialogu potvrďte tlačítkem '''OK''' [[Soubor:Accept.png|frameless|alt=List all|link=|]]. [[Soubor:Perun_smazani_uzivatele.png|link=|]] * Po stisku tlačítka '''OK''' uvidíte potvrzení ''Member: <name> deleted!'' [[Soubor:Perun_kontrola_smazaneho_uzivatele.png|link=|]] [[Kategorie:Pro správce]] [[Kategorie:Datové úložiště]] [[Kategorie:Česky]] [[Kategorie:Perun]] r9y7ocs806ffeqqnv3nnt6p67dxuwkl 0 124 364 363 2016-01-23T15:48:57Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Perun system - Sprava VO|Zpět na Perun system - Správa VO]] ---- <!--[[My role is VO MANAGER|''VO MANAGER]] ''role needed''--> * Podobně jako v procesu [[Schvalování uživatelů]] tak i při prodlužování členství Vám přijde email v obdobném formátu jako je uveden níže. V emailu najdete odkazy směřující na stránku s podrobnými informacemi o uživateli. Na odkazované stránce také máte možnost '''Schválit/Zamítnout''' přihlášku. Vyberte si preferovaný způsob přihlášení a přihlaste se tak do systému Perun pro zobrazení detailu žádosti o prodloužení. <pre> Vážený administrátore, byla podána nová žádost o prodloužení členství do "Název instituce" pod ID=46456457 uživatelem: "Jméno uživatele", "Email uživatele" / &#10;https://whoami.cesnet.cz/idp/shibboleth. Detail přihlášky s možností přijetí/zamítnutí: FEDERACE: https://einfra.cesnet.cz/perun-gui/#vo/appdetail?id=46456457 KERBEROS (@META): https://einfra.cesnet.cz/perun-gui-krb/#vo/appdetail?id=46456457 CERTIFIKÁT: https://einfra.cesnet.cz/perun-gui-cert/#vo/appdetail?id=46456457 Pokud při vytvoření přihlášky došlo k chybám, výpis následuje: -------------------------------------------------- Váš NGI Perun management team </pre> * Po kontrole přihlášky stiskněte tlačítko '''Approve''' [[Soubor:User_green.png|frameless|alt=Schválit|link=|]] pro schválení žádosti o prodloužení nebo tlačítko '''Reject''' [[Soubor:Reject.png|frameless|alt=Schválit|link=|]] pro zamítnutí žádosti. [[Kategorie:Pro správce]] [[Kategorie:Datové úložiště]] [[Kategorie:Česky]] [[Kategorie:Perun]] ee2k8ul3kc4d6u0k288yluw01wfu6kz 0 125 366 365 2016-01-23T15:48:57Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Perun system - Sprava VO|Zpět na Perun system - Správa VO]] ---- <!--[[My role is VO MANAGER|''VO MANAGER]] ''role needed''--> * Přihlaste se do prostředí [https://einfra.cesnet.cz/fed/gui/ Perun] a přejděte do sekce '''VO Manager''' [[File:Building.png|frameless|alt=VO|link=|]]. * V levém menu vyberte položku '''Select VO''', čímž zobrazíte všechny VO, které máte ve správě. * Ze seznamu vyberte požadovanou VO, ve které chcete vytvořit servisní účet. [[Soubor:Perun_gui.png|link=|]] * Nejjednodušší cestou, jak vytvořit servisní účet je zvolením tlačítka '''Create Service member''' na záložce '''Overview''' [[Soubor:Perun_vo.png|link=|]] * Objeví se dialogové okno '''1. Create service identity''', kde je potřeba vyplnit údaje o účtu: ** ''Member's name'' - Zde zadejte jméno servisního účtu ** ''Member's email'' - Email, který bude sloužit jako primární kontakt pro servisní identitu, pokud nechcete dostávat žádné maily, pak zadejte '''nomail@metacentrum.cz''', emaily odeslané na tuto adrtesu budou automaticky zahozeny. ** ''Namespace'' - Z rozbalovací nabídky zvolte namespace shodný s namespace uživatele, se kterým bude servisní účet spojený. ** ''Login'' - '''Uživatelské jméno volte tak, aby vždy začínalo názvem VO''', tj. např. servisní účet ''backup'' pro ''VO_meta'' bude mít login '''meta_backup''' [[Soubor:Perun_create_service_identity.png|link=|]] ** stiskněte tlačítko '''Continue''' * V následujícím dialogovém okně '''2. Associate real users''': ** do vyhledávacího pole zadejte jméno nebo login uživatele, se kterým bude servisní účet svázán, a stiskněte tlačítko '''Search''' [[Soubor:Perun_searched_users.png|link=|]] ** Zvolte uživatele označením zaškrtávacího pole u jména a stiskněte tlačítko '''Add''' [[Soubor:Perun_associate_real_users.png|link=|]] ** stiskněte tlačítko '''Continue''' * V dalším dialogovém okně '''3. Set password for: ''logname''''': ** vyplňte heslo do pole ''Password'' a stejnou hodnotu zadejte i do ''Re-type password'' [[Soubor:Perun_set_password.png|link=|]] ** Stiskněte tlačítko '''Set password''' * Servisní účet bude v tuto chvíli vytvořen a můžete ho najít v seznamu členů VO na záložce '''Members''' [[File:user_green.png|frameless|alt=VO|link=|]]. Servisní účty jsou označeny slovem ''(Service)'' [[File:user_red.png|frameless|alt=VO|link=|]] před jménem. [[Kategorie:Pro správce]] [[Kategorie:Datové úložiště]] [[Kategorie:Česky]] [[Kategorie:Perun]] 9axpyat38xspr0kwbxx8v8mjv4odn3f 0 126 368 367 2016-01-23T15:48:57Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Perun system - Sprava VO|Zpět na Perun system - Správa VO]] ---- <!--[[My role is VO MANAGER|''VO MANAGER]] ''role needed''--> * Přihlaste se do prostředí [https://einfra.cesnet.cz/fed/gui/ Perun] a přejděte do sekce '''User''' [[Soubor:User_gray.png|frameless|alt=User|link=|]]. [[Soubor:Perun_gui.png|link=|]] * V submenu sekce [[Soubor:User_gray.png|frameless|alt=User|link=|]]'''User''' v levém menu vyberte položku '''Service identities''' [[Soubor:User_red.png|frameless|alt=Service identities|link=|]]. * Zobrazí se Vám seznam všech servisních účtů, kterou jsou asociované s Vaším účtem. [[Soubor:Perun_switch_to_service_account.png|link=|]] * Zvolte požadovaný servisní účet. * Zobrazí se profil servisního účtu. * Na záložce '''Overview''' v sekci '''Quick links''' vyberte položku '''Change / reset password''', čímž přejdete na záložku '''Authentication'''. [[Soubor:Perun_change_password.png|link=|]] * Na záložce '''Authentication''' stiskněte tlačítko '''Change password''' pro změnu hesla. [[Soubor:Perun_change_password_2.png|link=|]] * Do zobrazeného formuláře vepiště bezpečné heslo do kolonky '''New password''' a pro kontrolu jej zopakujte v kolonce '''Retype new pass'''. [[Soubor:Perun_change_password_3.png|link=|]] * Po potvrzení tlačítkem '''Change password''' dojde ke změně hesla. Změny se ze systému Perun do spravovaných systému propagují v určitých intervalech, doporučujeme tedy vyčkat 30 - 60 minut než se pokusíte s účtem přihlašovat na dané služby. [[Kategorie:Pro správce]] [[Kategorie:Datové úložiště]] [[Kategorie:Česky]] [[Kategorie:Perun]] ghq364ormgdop7t91y1rcfqkd3oed9i 0 127 370 369 2016-01-23T15:48:58Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Perun system - Sprava VO|Zpět na Perun system - Správa VO]] ---- <!--[[My role is VO MANAGER|''VO MANAGER]] ''role needed''--> * Přihlaste se do prostředí [https://einfra.cesnet.cz/fed/gui/ Perun] a přejděte do sekce '''VO Manager''' [[File:Building.png|frameless|alt=VO|link=|]]. * V levém menu vyberte položku '''Select VO''', čímž zobrazíte všechny VO, které máte ve správě. * Ze seznamu vyberte požadovanou VO, do které chcete pozvat nové uživatele, otevře se nový panel s nastavením dané VO. [[Soubor:Perun_gui.png|link=|]] * Na záložce '''Overview''' klikněte na tlačítko '''Create group''' [[Soubor:Add.png|frameless|alt=invite_user|link=|]]. [[Soubor:Perun_vo.png|link=|]] * Zobrazí se dialogové okno, kde vyplňte: ** ''Name:'' Zadejte název skupiny, '''název volte tak, aby vždy začínalo názvem VO''', tj. např. skupina ''group'' pro ''VO_meta'' bude mít login '''meta_group'''. ** ''Descriptions:'' Zde můžete zadat vlastní popis skupiny, abyste odlišili její účel atp. ** ''As sub-group:'' Zaškrtněte, pokud chcete, aby skupina byla podskupinou jiné a v zobrazené rozbalovací nabídce ''Parent group'' zvolte skupinu, jejíž podskupinou má být. [[Soubor:Perun_create_group.png|link=|]] ** Potvrďte tlačítkem '''Create'''. '''V této fázi je připravená skupina, nyní je potřeba přiřadit dané skupině zdroje.''' * Přejděte na záložku '''Groups''' a vyberte skupinu, které chcete přiřadit zdroje. * Otevře se nastavení skupiny, zde přepněte na záložku '''Resources'''. [[Soubor:Perun_select_resources.png|link=|]] * Na této záložce stiskněte tlačítko '''[[Soubor:Add.png|frameless|alt=invite_user|link=|]] Add''' a ze zobrazeného seznamu vyberte požadované zdroje. ('''Vždy vybírejte zdroje s označením HOME.''' ''V případě Plzeňského úložiště vyberte „Storage Pilsen - home“, pro Jihlavu „Storage Jihlava - home“ a pro Brno „Storage Brno - home“'') [[Soubor:Perun_select_resources.png|link=|]] * Zdroj vyberete zaškrtnutím odpovídající zaškrtávacího pole a svou volbu potvrďte tlačítkem '''[[Soubor:Add.png|frameless|alt=invite_user|link=|]] Add'''. '''Nyní je potřeba ještě přiřadit skupině GID, aby mohlo dojít k propagaci skupiny na potřebné zdroje.''' * Pokud máte stále otevřenou záložku s nastavením skupiny, tak přejděte na záložku '''Settings'''. Pokud jste nastavení skupiny zavřeli, pak opět přejděte na záložku '''Groups''' v nastavení VO a vyberte skupinu, kterou potřebujete nastavit. [[Soubor:Perun_create_linux_group.png|link=|]] * Z rozbalovacího menu '''Resources''' zvolte Váš zdroj. V případě úložiště v Plzni '''Storage Pilsen - home''', pro Jihlavu '''Storage Jihlava - home''' a pro Brno '''Storage Brno - home'''. [[Soubor:Perun_create_linux_group_2.png|link=|]] * Stiskněte tlačítko '''[[Soubor:Add.png|frameless|alt=invite_user|link=|]] Add'''. * V seznamu proměnných najděte položku '''Unix group name in einfra''' a do pole vyplňte název skupiny, shodný se zvoleným názvem dříve a nastavení uložtě stisknutím tlačítka '''Save'''. * Po stisku tlačítka '''Save''' by se mělo vyplnit také pole '''Unix GID in einfra'''. '''Na závěr stačí plnit skupinu uživateli.''' * Pokud máte stále otevřenou záložku s nastavením skupiny, tak přejděte na záložku '''Members'''. Pokud jste nastavení skupiny zavřeli, pak opět přejděte na záložku '''Members''' v nastavení VO a vyberte skupinu, kam chcete přidat uživatele. [[Soubor:Perun_select_group_3.png|link=|]] * Stiskněte tlačítko '''Add''' a pomocí vyhledávacího pole najděte uživatele, kterého chcete přidat. [[Soubor:Perun_select_user.png|link=|]] * Až daného uživatele vyhledáte, zatrhněte políčku u jeho jména a stiskněte tlačítko '''Add'''. * Pokud chcete přidat více uživatelů, pokračujte opět vyhledáváním a přidání uživatele vždy potvrďte tlačítkem '''Add'''. [[Kategorie:Pro správce]] [[Kategorie:Datové úložiště]] [[Kategorie:Česky]] [[Kategorie:Perun]] nn0gstt9mil1lrmdo1fzlwm3tjo83on 0 128 372 371 2016-01-23T15:48:58Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Perun system - Sprava VO|Zpět na Perun system - Správa VO]] ---- <!--[[My role is VO MANAGER|''VO MANAGER]] ''role needed''--> * Přihlaste se do prostředí [https://einfra.cesnet.cz/fed/gui/ Perun] a přejděte do sekce '''VO Manager''' [[File:Building.png|frameless|alt=VO|link=|]]. [[Soubor:Perun_gui.png|link=|]] * V levém menu vyberte položku '''Select VO''', čímž zobrazíte všechny VO, které máte ve správě. * Ze seznamu vyberte požadovanou VO, do které chcete pozvat nové uživatele, otevře se nový panel s nastavením dané VO. [[Soubor:Perun_vo.png|link=|]] * Přejděte na záložku '''Members''' a vyhledejte uživatele pomocí vyhledávacího pole, nebo použijte tlačítko '''List all''' pro zobrazení všech uživatelů. [[Soubor:Perun_members.png|link=|]] [[Soubor:Perun_list_all.png|link=|]] * Vyberte uživatele, kterému chcete nastavit kvótu, čímž se Vám otevře záložka s nastavením uživatele. [[Soubor:Perun_account_overview.png|link=|]] * Přejděte na záložku '''Settings'''. [[Soubor:Perun_account_setting.png|link=|]] * Na této kartě vyberte z rozbalovací nabídky '''Assigned resources''' zdroj, na kterém chcete kvóty nastavit. [[Soubor:Perun_account_assigned_resources.png|link=|]] * Položky jsou vysvětleny níže v tabulce. {| class="wikitable" |Data limit |Jedná se o tzv. hard kvótu, což je limit, který nelze překročit. Zadává se včetně jednotek (M,G,T, …), G je výchozí. Pokud tuto hodnotu vyplníte, tak definujete vyjímku oproti globálnímu nastavení u zdroje (resource). |- |Data quota |Jedná se o tzv. soft kvótu, což je limit, který lze dočasně překročit. O jejím překročení bude uživatel informován. Zadává se včetně jednotek (M,G,T, …), G je výchozí. Pokud tuto hodnotu vyplníte, tak definujete vyjímku oproti globálnímu nastavení u zdroje (resource). |- |Files limit |Hard quota určuje maximální počet souborů, které může uživatel nahrát. Pokud tuto hodnotu vyplníte, tak definujete vyjímku oproti globálnímu nastavení u zdroje (resource). |- |Files quota |Soft quota určuje maximální počet souborů, které může uživatel nahrát. Pokud tuto hodnotu vyplníte, tak definujete vyjímku oproti globálnímu nastavení u zdroje (resource). |} [[Kategorie:Pro správce]] [[Kategorie:Datové úložiště]] [[Kategorie:Česky]] [[Kategorie:Perun]] m8920htm3o3c9c4uxfyegt3mdw6yu63 0 129 374 373 2016-01-23T15:48:58Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Perun system - Sprava VO|Zpět na Perun system - Správa VO]] ---- <!--[[My role is VO MANAGER|''VO MANAGER]] ''role needed''--> * Přihlaste se do prostředí [https://einfra.cesnet.cz/fed/gui/ Perun] a přejděte do sekce '''VO Manager''' [[File:Building.png|frameless|alt=VO|link=|]]. [[Soubor:Perun_gui.png|link=|]] * Přejděte do sekce [[Soubor:User_gray.png|frameless|alt=User|link=|]]'''User''' a vyberte položku [[Soubor:User_red.png|frameless|alt=Service identities|link=|]]'''Service identities'''. Zobrazí se Vám seznam Vašich servisních účtů. [[Soubor:Perun_switch_to_service_account.png|link=|]] * Vyberte patřičnou servisní identitu. (Poznámka: [[Soubor:User_red.png|frameless|alt=Service identities|link=|]]'''Service identities''' se automaticky změní na [[Soubor:User_red.png|frameless|alt=Service identities|link=|]]'''Associated users'''. [[Soubor:Perun_profile_service_account.png|link=|]] * Z menu zvolte [[Soubor:User_red.png|frameless|alt=Service identities|link=|]]'''Associated users''' a následně [[Soubor:Add.png|frameless|alt=invite_user|link=|]]'''Add'''. [[Soubor:Perun_associate_real_users_2.png|link=|]] * Zadejte do vyhledávacího pole jméno, příjmení, email nebo login uživatele, na kterého chcete asociovat vytvořený servisní účet a stiskněte tlačítko [[Soubor:Find.png|frameless|alt=List all|link=|]]'''Search'''. [[Soubor:Perun_searched_users_2.png|link=|]] * Z výsledků vyhledávání vyberte Vašeho uživatele a zvolte [[Soubor:Add.png|frameless|alt=invite_user|link=|]]'''Connect'''. [[Kategorie:Pro správce]] [[Kategorie:Datové úložiště]] [[Kategorie:Česky]] [[Kategorie:Perun]] h7p3domoc4ftnoktenhs5dnp7vnfyur 0 138 387 2016-01-23T16:16:32Z Root 1 Založena nová stránka s textem „External sources are used to get information about users from external systems. Perun has several predefined connectors: LDAP and SQL. External source defi…“ wikitext text/x-wiki External sources are used to get information about users from external systems. Perun has several predefined connectors: LDAP and SQL. External source defines how to connect to the external system (url, hostname, login, password, ...) and what are the transformation rules between external system and Perun. == Perun external sources definition == Already implemented: * cz.metacentrum.perun.core.impl.ExtSourceXML * cz.metacentrum.perun.core.impl.ExtSourceLdap * cz.metacentrum.perun.core.impl.ExtSourceSql * cz.metacentrum.perun.core.impl.ExtSourceISMU * cz.metacentrum.perun.core.impl.ExtSourceCSV * cz.metacentrum.perun.core.impl.ExtSourceGoogle After a change of the configuration file with external resources "/etc/perunv3/perun-extSources.xml", command"./loadExtSourcesDefinitions" must be run to reload configuration file. === XML === IMPORTANT: In queries part (?) can't be used character ['] cause problem of escaping (for now) Type: cz.metacentrum.perun.core.impl.ExtSourceXML Connection parameters (for local file): * '''file''' - path to file with xml (local) e.g. /etc/perun/file.xml (if file is filled, use file) Connection parameters (for external XML file from web aplication): * '''uri''' - uri where xml exists e.g. https://something.cz/getListOfMembers (if file is not filled, uri must be filled and then is used) * '''keyStore''' - location of keystore with certificate e.g. /etc/perun/perun.p12 (for ssl comunicaton) * '''keyStorePass''' - password for keystore certificate e.g. password (for ssl comunicaton) * '''keyStoreType''' - type of keystore certificate e.g. PKCS12 (for ssl comunicaton, PKCS12 is recomended) * '''trustStore''' - trust store location with certificates of server e.g. /etc/perun/truststore (for ssl comunicaton) * '''trustStorePass''' - password to trustStore certificates e.g. password (for ssl comunicaton) Other parameters: * '''xpath''' - xpath query where ? will be assigned required value e.g. //member[starts-with(DN/text(),'?') * '''loginXpath''' - xpath query returning particular record according to the user login. It must return one or no record, e.g. //member[DN/text()='?'] * '''xmlMapping''' - pairing of attribute name in Perun and xpath query, e.g. login=./DN/text(). As a value must be local xpath query or regex with xpathQuery ** local query - ./DN/text() ** xpath + regex - #[regex]/[replacement]#[xpath query] e.g. #^.*(CN|cn)([abc]+)$/$2#./DN/text() ** when we want to add another userExtSources to user, we use following parameter. First add more userExtSources, then add numbering. Syntax: additionalues_[no.]=concat([ExtSource name],'|[ExtSource type]|', [Pattern for extLogin], '|[LoA]') UserExtSource example: additionalues_1=concat(./CA/text(),'|cz.metacentrum.perun.core.impl.ExtSourceX509|', ./DN/text(), '|2' ) XML external source example: <extSource> <name>BIOMEDVOMS</name> <type>cz.metacentrum.perun.core.impl.ExtSourceXML</type> <description></description> <attributes> <attribute name="keyStore">/ExtSourceVoms/perun-cesnet-cz.p12</attribute> <attribute name="keyStorePass">pass</attribute> <attribute name="keyStoreType">PKCS12</attribute> <attribute name="trustStore">/ExtSourceVoms/trustStore</attribute> <attribute name="trustStorePass">pass</attribute> <!-- If file is filled, use file. If not, use uri.--> <attribute name="uri">https://cclcgvomsl.cz/listMembers</attribute> <attribute name="file"></attribute> <attribute name="xpath">//multiRef[starts-with(DN/text(),'/') and contains(DN/text(),'?')]</attribute> <attribute name="loginXpath">//multiRef[starts-with(DN/text(),'/') and DN/text()='?']</attribute> <attribute name="xmlMapping"> additionalues_1=concat(./CA/text(),'|cz.metacentrum.perun.core.impl.ExtSourceX509|', ./DN/text(), '|2' ), login=./DN/text(), urn:perun:member:attribute-def:def:mail=./mail/text(), firstName=#^.*(CN|cn)=([0-9]*\s*)(([a-zA-Z]*)[- .])?([-a-zA-Z. ']*)([ ,\/].*)?$/$4#./DN/text(), lastName=#^.*(CN|cn)=([0-9]*\s*)(([a-zA-Z]*)[- .])?([-a-zA-Z. ']*)([ ,\/].*)?$/$5#./DN/text() </attribute> </attributes> </extSource> === LDAP === Type: cz.metacentrum.perun.core.impl.ExtSourceLdap Connection parameters: * '''url''' - e.g. ldap://meta-ldap.cesnet.cz:389/ou=People,o=meta,c=cz * '''user''' - e.g. CN=test,OU=Users,O=perun * '''password''' * '''referral''' - e.g. follow * '''query''' - query template to LDAP, as ? will be assigned required value e.g. {{{(uid=*?*)}}} or {{{(CN=?)}}} * '''loginQuery''' - query template to LDAP, returning particular record according to the user login. It must return one or no record, e.g. {{{(CN=?}}} * '''ldapMapping''' - pairing of attribute name in Perun and item name in LDAP, e.g. firstName={givenName}. Item name in LDAP must be closed into {}. As a value could be used constant or a combination with LDAP item, for example login={uid}@muni.cz ** when attribute in LDAP contains one or more value but only one is wanted, use index. For example: {mail[0]}, take the first of mails. ** when attribute in LDAP contains more values, that must be mapped to more attributes in Perun, regular expressions can be used:[perunAttrName]={[ldapAttrName]|[regex]}, for example: attribute "title" in LDAP contains "RNDr.:PhD." into ldapMapping we put two definitions <pre> {{{ titleBefore={title|^(.*):.*},titleAfter={title|^.*:(.*)} }}} </pre> Value in "()" in regexp will be taken to each definition. ** when we want to add another userExtSources to user, we use following parameter. First add more userExtSources, then add numbering. Syntax: additionalues_[no.]=[ExtSource name]|[ExtSource type]|[Pattern for extLogin]|[LoA] UserExtSource example: additionalues_1=https://idp2.ics.muni.cz/idp/shibboleth|cz.metacentrum.perun.core.impl.ExtSourceIdp|{cn}@muni.cz additionalues_2=IS.MUNI.CZ|cz.metacentrum.perun.core.impl.ExtSourceKerberos|{cn}@IS.MUNI.CZ LDAP external source example: <extSource> <name>LDAPMU</name> <type>cz.metacentrum.perun.core.impl.ExtSourceLdap</type> <description></description> <attributes> <attribute name="url">ldap://server</attribute> <attribute name="base">OU=Users,OU=Org,DC=us</attribute> <attribute name="query">(CN=?)</attribute> <attribute name="loginQuery">(CN=?)</attribute> <attribute name="user">CN=admin,OU=System,DC=us</attribute> <attribute name="password">bla</attribute> <attribute name="referral">follow</attribute> <attribute name="ldapMapping"> firstName={givenName}, lastName={sn}, login={cn}, urn:perun:member:attribute-def:def:mail={mail[0]}, urn:perun:member:attribute-def:def:organization=My organization, urn:perun:user:attribute-def:def:login-namespace:mu={cn}, titleBefore={title|^(.*):.*}, titleAfter={title|^.*:(.*)}, additionalues_1=https://idp.org.us/idp/shibboleth|cz.metacentrum.perun.core.impl.ExtSourceIdp|{cn}@org.us|2 </attribute> </attributes> </extSource> === SQL === Type: cz.metacentrum.perun.core.impl.ExtSourceSql Connection parameters: * '''driver''' - selecting DB driver e.g. com.mysql.jdbc.Driver * '''url''' - JDBC url, e.g.. jdbc:mysql://localhost:3307/DB * '''user''' * '''password''' * '''query''' - This query is used to find person record in external DB, it can be used for fulltext search. Column name must match with attribute definition in Perun. The '?' is then replaced by the search string, e.g. when you are searching user while adding into the VO. Because some of the DB engines has limitation on column name length, attribute names have to be shorted by following way - '''x:y:z''' : {| style="border: solid 1px grey;" cellpadding="5" cellspacing="0" | style="font-weight: bold; background-color: lightgrey;" | x || style="font-weight: bold; background-color: lightgrey;" | means |- |m || member |- |u || user |- |f || facility |- |r || resource |- |g || group |- |v || vo |- |h || host |- |mr || member_resource |- |uf || user_facility |- |gr || group_resource |} {| style="border: solid 1px grey;" cellpadding="5" cellspacing="0" |style="font-weight: bold; background-color: lightgrey;" | y || style="font-weight: bold; background-color: lightgrey;" | means |- |d || def |- |o || opt |} '''z''' - means value of attribute ''Examples of shorting of attributes'': m:d:address -> urn:perun:member:attribute-def:def:address u:d:login-namespace:mu -> urn:perun:user:attribute-def:def:login-namespace:m For attributes firstName, lastName, middleName, titleBefore, titleAfter and login, you do not need to specify attribute name, these are predefined. For example: <pre> {{{ select name as first_name, surname as last_name, login, mail as "m:d:mail", telephone as "m:d:phone", logname as login from someTable where (first_name || ' ' || last_name) like '%' || ? || '%' }}} </pre> Will do the fulltext search for the user, matching [first_name last_name]. * '''loginQuery''' - This query is used to get information about particular person, login returned by 'query' will be used as a replacement for '?' in loginQuery. The loginQuery must return one or no record. For example <pre> {{{ select first_name as firstName, last_name as lastName, '' as middleName, title as titleBefore, '' as titleAfter, mail as "m:d:mail", logname as login from someTable where logname=? }}} </pre> ** When we want to assign another userExtSources to user, we use following syntax Syntax: [ExtSource name]|[ExtSource type]|[Pattern for extLogin]|[LoA] as additionalues_[no.] For example, just add it to the select query (MySQL example): concat(ca.ca,'|cz.metacentrum.perun.core.impl.ExtSourceX509|',usr.dn,'|',2) as `additionalues_1 concat("EINFRA",'|cz.metacentrum.perun.core.impl.ExtSourceKereros|',usr.login,'|',2) as `additionalues_2 Example of SQL external source (MySQL): <extSource> <name>BLA</name> <type>cz.metacentrum.perun.core.impl.ExtSourceSql</type> <description></description> <attributes> <attribute name="driver">com.mysql.jdbc.Driver</attribute> <attribute name="url">jdbc:mysql://localhost:3307/DB</attribute> <attribute name="query"> select usr.userid as `login`, SUBSTRING_INDEX(cn, ' ',1) as `firstName`, (case SUBSTRING(cn, length(SUBSTRING_INDEX(cn, ' ',1))+2) when '' then 'N/A' else SUBSTRING(cn, length(SUBSTRING_INDEX(cn, ' ',1))+2) end) as `lastName`, usr.mail as `m:d:mail`, usr.mail as `u:d:preferredMail`, concat(ca.ca,'|cz.metacentrum.perun.core.impl.ExtSourceX509|',usr.dn,'|',2) as `additionalues_1` from usr, ca where usr.userid=? and usr.ca=ca.cid </attribute> <attribute name="loginQuery"> select usr.userid as `login`, SUBSTRING_INDEX(cn, ' ',1) as `firstName`, (case SUBSTRING(cn, length(SUBSTRING_INDEX(cn, ' ',1))+2) when '' then 'N/A' else SUBSTRING(cn, length(SUBSTRING_INDEX(cn, ' ',1))+2) end) as `lastName`, usr.mail as `m:d:mail`, usr.mail as `u:d:preferredMail`, concat(ca.ca,'|cz.metacentrum.perun.core.impl.ExtSourceX509|',usr.dn,'|',2) as `additionalues_1` from usr, ca where usr.userid=? and usr.ca=ca.cid </attribute> <attribute name="user">admin</attribute> <attribute name="password">admin</attribute> </attributes> </extSource> === IS MU === Type: cz.metacentrum.perun.core.impl.ExtSourceISMU Used for synchronization with IS Masaryk University. Connection parameters: * '''url''' - URL to define members in group, for example. https://is.muni.cz/auth/export/skupina_osob.pl?format=csv&amp;kodovani=utf-8 * '''user''' * '''password''' === CSV === Type: cz.metacentrum.perun.core.impl.ExtSourceCSV Connection parameters: * '''file''' - path to CSV file (local) e.g. /etc/perun/file.csv, in this case CSV file has to contain header on the first row * '''query''' - query for finding users (rows in CSV file) with given attributes, e.g. ''login contains ?'', where attribute login represents column name from CSV header and ''contains'' is a key word that is mandatory for query parameter * '''loginQuery''' - query for finding a user (row in CSV file) with given primary attribute, e.g. ''eppn=?'', where attribute eppn represents column name from CSV header * '''csvMapping''' - pairing of attribute name in Perun and values from CSV file, e.g. urn:perun:facility:attribute-def:def:login-namespace={login} CSV file example: login,eppn,email xuser1,123456@muni.cz,xuser1@mail.muni.cz CSV external source example: <extSource> <name>CSV</name> <type>cz.metacentrum.perun.core.impl.ExtSourceCSV</type> <description></description> <attributes> <attribute name="file">/etc/perun/users.csv</attribute> <attribute name="query">login contains ?</attribute> <attribute name="loginQuery">eppn=?</attribute> <attribute name="csvMapping"> urn:perun:user:attribute-def:core:firstName={first_name}, urn:perun:user:attribute-def:core:lastName={last_name}, urn:perun:facility:attribute-def:def:login-namespace={login}, eppn={eppn} </attribute> </attributes> </extSource> === Google === Type: cz.metacentrum.perun.core.impl.ExtSourceGoogle Connection parameters: * '''domain''' - domain url, where the users come from * '''group''' - Google group name in domain, from which the users are imported to Perun. It has to contain whole address of the group, that means something like this: ''name1@einfra.cesnet.cz'' (including '@domainName) * '''query''' - query for finding users in given Google group with given attributes, e.g. ''email contains ?'', where ''contains'' is a key word that is mandatory for query parameter. It is possible to query only 'email' parameter * '''loginQuery''' - query for finding a user in given Google group with given primary attribute, e.g. ''id=?''. It is possible to query only 'email' and 'id' parameters. * '''serviceAccountEmail''' - an email generated according to [https://developers.google.com/identity/protocols/OAuth2ServiceAccount#creatinganaccount this guide]. * '''userEmail''' - an email address of the user in the domain. Application will act like this user and will query for users in specified group. * '''p12File''' - path to the file containing P12 key generated according to guide mentioned above * '''googleMapping''' - pairing of attribute name in Perun and value from Google group, e.g. urn:perun:user:attribute-def:virt:logins-namespace:google={userID}. Only {userID}, {domainName} and {groupName} can be connected like this, all other attributes in Perun have to be connected with exact value, e.g. urn:perun:user:attribute-def:def:preferredMail=name@gmail.com. Google external source example: <extSource> <name>GOOGLE</name> <type>cz.metacentrum.perun.core.impl.ExtSourceGoogle</type> <description></description> <attributes> <attribute name="domain">einfra.cesnet.cz</attribute> <attribute name="group">name1@einfra.cesnet.cz</attribute> <attribute name="query">email contains ?</attribute> <attribute name="loginQuery">id=?</attribute> <attribute name="serviceAccountEmail">id@developer.gserviceaccount.com</attribute> <attribute name="userEmail">login@einfra.cesnet.cz</attribute> <attribute name="p12File">/etc/perun/client_secret_pkcs12.p12</attribute> <attribute name="googleMapping"> urn:perun:user:attribute-def:virt:logins-namespace:google={userID}, urn:perun:facility:attribute-def:def:googleGroupNameNamespace={domainName}, urn:perun:group:attribute-def:def:googleGroupName-namespace:einfra.cesnet.cz={groupName}, urn:perun:user:attribute-def:def:preferredMail=name@gmail.com </attribute> </attributes> </extSource> [[Category:Perun]] hz4x02o9cdrecukkchimigk1y7wwc6f 0 139 388 2016-01-23T16:18:03Z Root 1 Založena nová stránka s textem „Group synchronization allows to fill the Perun's group with the users from external system. In order to use group synchronization, you have to have properl…“ wikitext text/x-wiki Group synchronization allows to fill the Perun's group with the users from external system. In order to use group synchronization, you have to have properly setup external sources [[Perun_external_sources]] which define how to connect to the external source. To the group which we need to synchronize with external source we have to set following attributes: * '''urn:perun:group:attribute-def:def:synchronizationEnabled''' - required values are ''true/false'' * '''urn:perun:group:attribute-def:def:synchronizationInterval''' - it must be set in multiples of 5 minutes * '''urn:perun:group:attribute-def:def:groupExtSource''' - name of [[Perun external sources|external source]] from which are getting data about group, list of logins which can be than searched in ''groupMembersExtSource'' for getting information about each member. * '''urn:perun:group:attribute-def:def:groupMembersExtSource''' - name of external source from which are getting data about members of group. Attribute is ''optional'' it is used only if one external source provides information about groups and different one provides data about group members * '''urn:perun:group:attribute-def:def:groupMembersQuery''' - Query to the external source which must return only one column called login. == SQL Database == * Example of groupMembersQuery for external source of type SQL database ''cz.metacentrum.perun.core.impl.ExtSourceSql'', this query must return one column called login. select logname as login from someTable where theirsGroupId=123 == SQL Query == SQL query returns column '''firstName''', '''lastName''', '''login'''. Optionally ''middleName'', ''titleBefore'', ''titleAfter''. SQL query could return columns with names of concrete attributes. Due to size restriction, attribute names must be shorten, where x:y:z means: {| style="border: solid 1px grey;" cellpadding="5" cellspacing="0" | style="font-weight: bold; background-color: lightgrey;" | x || style="font-weight: bold; background-color: lightgrey;" | meaning |- |m || member |- |u || user |- |f || facility |- |r || resource |- |g || group |- |v || vo |- |h || host |- |mr || member_resource |- |uf || user_facility |- |gr || group_resource |} {| style="border: solid 1px grey;" cellpadding="5" cellspacing="0" |style="font-weight: bold; background-color: lightgrey;" | y || style="font-weight: bold; background-color: lightgrey;" | meaning |- |d || def |- |o || opt |} Examples: m:d:address -> urn:perun:member:attribute-def:def:address u:d:login-namespace:mu -> urn:perun:user:attribute-def:def:login-namespace:mu * Example groupMembersQuery for external resource SQL database ''cz.metacentrum.perun.core.impl.ExtSourceSql'' select name as first_name, surname as last_name, login, mail as "m:d:mail", telephone as "m:d:phone" from someTable * Example groupMembersQuery for external resource LDAP ''cz.metacentrum.perun.core.impl.ExtSourceLdap'' TBA * Example groupMembersQuery for external resource ISMU ''cz.metacentrum.perun.core.impl.ExtSourceISMU'' https://is.muni.cz/auth/export/skupina_osob.pl?format=csv&kodovani=utf-8&skup_zkratka=1433:15 == LDAP == * Example groupMembersQuery for external source of type LDAP ''cz.metacentrum.perun.core.impl.ExtSourceLdap'' (gropuId=123) == IS MU == * Example of groupMembersQuery for external source of type ISMU ''cz.metacentrum.perun.core.impl.ExtSourceISMU'' https://is.muni.cz/auth/export/skupina_osob.pl?format=csv&kodovani=utf-8&skup_zkratka=1433:15 == Synchronization with the other Perun group == Group members could be synchronized with other group. As external resource is used the with the name ''''PERUN''''. Attributes must be filled in the followin way: * urn:perun:group:attribute-def:def:synchronizationEnabled set '''true/false''' depending on manual or automatic synchronization * urn:perun:group:attribute-def:def:synchronizationInterval fill required value. When value in preceding attribute is '''true''', multiple this value by 5 minutes. * urn:perun:group:attribute-def:def:groupExtSource set to '''PERUN''' * urn:perun:group:attribute-def:def:groupMembersQuery set to '''select members.user_id as login from members, groups_members where groups_members.group_id=[id skupiny] and members.id=groups_members.member_id''' Syncing validates synchronized members. To get only VALID members add '''and members.status=0''' to the membersQuery. == Example == * How to create group synchronized with external source of type SQL, in 5 minutes interval: groupExtSource = PERUNPEOPLE groupMembersQuery = 'select logname as login from someTable where groupId=123' synchronizationEnabled = true synchronizationInterval = 1 * Fire up synchronization manually using CLI: ./synchronizeGroup --groupId 123 * The same task using CLI: ./setGroupAttribute --groupId 123 --attributeName 'urn:perun:group:attribute-def:def:groupExtSource' --attributeValue PERUNPEOPLE ./setGroupAttribute --groupId 123 --attributeName 'urn:perun:group:attribute-def:def:groupMembersQuery' --attributeValue 'select name as first_name, surname as last_name, login, mail as urn:perun:member:attribute-def:def:mail, telephone as urn:perun:member:attribute-def:def:phone from someTable' ./setGroupAttribute --groupId 123 --attributeName 'urn:perun:group:attribute-def:def:synchronizationEnabled' --attributeValue true ./setGroupAttribute --groupId 123 --attributeName 'urn:perun:group:attribute-def:def:synchronizationInterval' --attributeValue 1 Manual synchronization in CLI: ./synchronizeGroup --groupId 123 == Example == *The example shows how to add users to group. Users are from VO MetaCentrum and their organization is 'Masarykova univerzita'. As external resource is used ''''PERUN''''. * urn:perun:group:attribute-def:def:synchronizationEnabled set '''true/false''' depending on manual or automatic synchronization * urn:perun:group:attribute-def:def:synchronizationInterval fill required value. When value in preceding attribute is '''true''', multiple this value by 5 minutes. * urn:perun:group:attribute-def:def:groupExtSource set to '''PERUN''' * urn:perun:group:attribute-def:def:groupMembersQuery set to '''select users.id as login from members, users, user_attr_values where user_attr_values.user_id=users.id and user_attr_values.attr_id=1363 and user_attr_values.attr_value='Masarykova univerzita' and members.vo_id=21 and members.user_id=users.id''' [[Category:Perun]] axexihrzbvr19egpgzg7dxsbjvrv3t0 0 140 389 2016-01-23T16:28:15Z Root 1 Založena nová stránka s textem „{{Template:Perun-en-template}} == Directly controlled functions == === User's accounts === They are controlled by perun services ''group'', ''fs_scratch'…“ wikitext text/x-wiki {{Template:Perun-en-template}} == Directly controlled functions == === User's accounts === They are controlled by perun services ''group'', ''fs_scratch'' and ''passord''. Files created by services are propagated to computational nodes and more processed. ''Mailaliases'' service belongs to these services too. It controls creation of mail aliases at target node. === Home directories === Home directories controls service ''fs_home'' to nodes where they have to be created. Related with this service is ''k5login'' service which maintains ".k5login" files in home directories. Home directories are created on storages and several frontends. === Project directories === They are created by service ''fs_project'' at Data storages current in Jihlava. Groups which can acces to project directory need to be assigned to resource target to project directory. === Replications === Replications are created by ''fs_replicas'' service. Currently is run at nodes "store1.du1/2/3.cesnet.cz" facilities "fe.du1/2/3.cesnet.cz". === NFS4 access === Is controlled by ''passwd_nfs4'', ''group_nfs4'' on storages. === Root access === Is realized by ''k5login_root'' service which is usually assigned to the special resource created for this purpose. This is concerned computationl nodes, storages and operational machines too. By a simillar way is realized service ''sshkeys_root'' which kept ssh keys with root access at selected machines. === SSH keys === Service ''sshkeys'' is target to mantain of files of users ssh keys. Similarly service ''sshkeys_root'' controles file with ssh keys for root access. === AFS groups and their members === These functions are controlled by services ''afs'' and ''afs_group''. Currently are used on facility AFS-ICS. === Eduroam === Service ''eduroam_radius'' creates list of eduroam identities. Currently it is located on facilities "eduroam" and "radius.ics.muni.cz". === PBS monitor and other information for PBS === PBS and PBS monitor is supported by services ''pbsmon_json'', ''pbs_phys_cluster'', ''pbsmon_users'', ''pbs_pre'' and ''pbs_publication_fairshare''. Service ''pbs_pre'' marked computational nodes as controlled by PBS or torque, real propagation of information about users is executed on "arien.ics.muni.cz", "wagap.cerit-sc.cz" and "zkusebni_planovac". Similarly goes service ''pbs_phys_cluster'' too, which works on "arien.ics.muni.cz" and "wagap.cerit-sc.cz". Service ''pbsmon_json'' transfer information about accessible machines and clusters and it is executed on "segin.ics.muni.cz". Services ''pbsmon_users'' and ''pbs_publication_fairshare'' provide to PBS information about users, second one with regard to publications; ''pbsmon_users'' is executed on "segin.ics.muni.cz", ''pbs_publication_fairshare'' is propagated to "arien.ics.muni.cz" and "wagap.cerit-sc.cz". === Information system of CESNET === Service ''users_export'' selects information about users from DB and propagates them to IS. As target of propagation is created facility "Informacni system CESNET". === VOMS === Service ''voms'' controlles VOMS server for South African Grid - Catch All VO, for EGI, VOCE, Auger VOs by facilities "voms1.egee.cesnet.cz" and "voms2.grid.cesnet.cz". === OPENVPN === Service ''openvpn'' generates list of IGTF certificates of users who are permitted to use OpenVPN system. Currently service is not realized. === For fedcloud === Service ''fedcloud_export'' ensures updating of users data in fedcloud infrastructure. Service is propagated to facilities which names are started with "egi-fedcloud-" === Access to licence server Flexlm === Access is realized by ''flexlm_iptables'' service based on IP address. Service is placed at "skirit.ics.muni.cz", "skiritf.ics.muni.cz" and "lm.zcu.cz", but currently is not used. === Postal services === Service ''mailman'' upkeeps members of mailinglists in software Mailman. Special version of service for MetaCenter is named ''mailman_meta''. Service ''mailman_owners'' controls list of managers of mailnglists in software Mailman. Service ''sympa'' has simillar function as ''mailman'' but it doesn't cooperate with Mailman software. === Apache === Files for Apache controls service ''apache_basic_auth'' which adds or deletes data in file Apache basic auth. The service is realized at facility "projekty.ics.muni.cz". Next service ''apache_ssl'' creates list of DN of certificates of users, who are permitted for access to specific directory and propagates it to Apache configuration. Service is placed at facilities "naiglos.ics.muni.cz", "aiglos.zcu.cz" and "pakiti.ics.muni.cz". === Clouds === For clouds are created following sercices: * ''fedcloud_export'' - service updates users in fedcloud infrastructure. It is realized at facilities "egi-fedcloud-infn-ct","egi-fedcloud-sztaki","egi-fedcloud-cesnet","egi-fedcloud-gwdg" and "egi-fedcloud-i3m-upv". * ''metacloud_export'' - special version of previous service for MetaCenter. It is placed at "carach.ics.muni.cz" facility. * ''owncloud_vo_mapping'' - special service for OwnCloud instance of MetaCenter, where mapping of users to virtual organisations is needed. It is assigned to the same facilities as ''fedcloud_export'' service. * ''cloudidp'' - service for ???, realized at facility "CloudIdP at GARR". === LDAPs === * Special functionality is realized beyond perun services. It pushes flat structure of perun to Perun LDAP, where it is usable for other different cooperating systems. * Services ''ldap_ad_ceitec'' and ''ldap_vsb_vi''. The first one is prepared at MU instance of Perun but it has not been used yet. The second one is ready to use for arrangement of access to VŠB tools but it has not been used yet too. === Gridmaps === Service ''gridmap'' creates GRIDMAPFILEs for MetaCenter an KYPO. It is placed at facility facility "metalb.ics.muni.cz". === Data repositories === Service ''du_users_export'' is special export of users for CESNETs purposes. Service is realized at "fe.du1.cesnet.cz","fe.du2.cesnet.cz" a "fe.du3.cesnet.cz" facilities. Service ''samba_du'' is special version of service ''samba'' for Data reporitories of CESNET. It is placed at facility "ldap.du2.cesnet.cz". === Documents, libraries etc. === Service ''docdb'' controls access to DocDB document server. It is document database for CERIT-SC. Service is realized at facility "DocDB-CERIT" with destination at "marach.ics.muni.cz". Service ''redmine-MU'' controls access to redmine of MU. It is placed at "Redmine-UVT", updates https://projekty2.ics.muni.cz/. === Samba === Service ''samba_du'' controls access to shared filesystem SAMBA. It is placed at facility "ldap.du2.cesnet.cz". === Kerberos === Service ''pkinit'' passes user logins including realms on Kerberos. All DNs of user certificates including X509 external identities passes on Kerberos too. Service is realized at "naiglos.ics.muni.cz". === Hadoop === Services ''hadoop_base'' and ''hadoop_hdfs'' controls access to HADOOP server, which allowes fast working with extremely big data. Services are placed at node "hador-c1.ics.muni.cz" at facility "hador-cluster.ics.muni.cz". === CEITEC === * Service ''labkey'' controls LabKey server for group Proteomics of project CEITEC. It is realized at facility "labkey.ics.muni.cz". * Service ''ldap_ad_ceitec'' creates LDAP for project CEITEC (see [[What can control Perun#LDAPs|LDAPs]]). === EGI === Service ''appDB'' updates "appDB" database of users for EGI. It is realized at facility "appDB.egi.eu" on host "perun.metacentrum.cz". == Indirectly controlled functions == === Accesses === * Access to "meetings.cesnet.cz" (system for booking and controlling of videoconferencial resources) - it is realized by perun's API. Service user especially created for CESNET directly asks to Perun by RPC. If needed LDAP can be asked too. ''Example:'' user from ÚVT MU has to be a member of "uvt" VO and a member in group "projects:shongo:users:uvt" in VO "einfra" simultaneously. * accesses to RT - they are realized by LDAP which is created by Perun (see [[What can control Perun#Ldaps|LDAPs]]). Currently are in Perun created groups for each queue which name starts with "RT" and these are selected by RT system from LDAP. More systemic solution is preparing. === Perun works as IdP and attribut authority === Attribut authority is used for: * consolidation of user identities * membership in groups through different organizations * authorization in Apache * authorization in DokuWiki * configuration of Shibboleth SP * authentication at Mailman for MetaCenter * authentication at Mailman for CERIT-SC * authentication at DocDb for CERIT-SC [[Category:Perun]] jh1outpzuhl9sg7p2zsp5oaa0rvmz9d 0 141 390 2016-01-23T16:29:19Z Root 1 Založena nová stránka s textem „ * You might encounter these bugs when using Perun, which are not about to be fixed anytime soon. * When using Perun with Oracle DB backend, you might enc…“ wikitext text/x-wiki * You might encounter these bugs when using Perun, which are not about to be fixed anytime soon. * When using Perun with Oracle DB backend, you might encounter errors when storing non-ASCII (like UTF-8) strings longer than 1000 characters. Storing bigger loads of textual data like certificate is not affected, since it's not stored as simple string. * When using Perun with Postgre DB and storing attribute values via CLI or programatically you can store empty string values, which are not allowed. When using GUI, this is not the case, since it handles input by itslef. On Oracle DB empty strings are stored as nulls by DB design, therfore when read, handled like "attribute not set" by Perun. But on Postgre DB is value stored as valid, therefore handled by Perun like "attribute set with value". This can cause unknown errors, especially when service configuration is about to be propagated to your resources. [[Category:Perun]] teuyw49cv0qd0jqlz4vxkgi5ag77u9w 0 592 2117 2016-05-12T17:02:10Z Root 1 Založena nová stránka s textem „[[Use cases|Back to Use cases]].....OR.....[[Less frequent tasks|Back to Less frequent tasks]] {{info|This text should serve as recommendation how to stru…“ wikitext text/x-wiki [[Use cases|Back to Use cases]].....OR.....[[Less frequent tasks|Back to Less frequent tasks]] {{info|This text should serve as recommendation how to structure system of groups in your VO: '''Good example''': You identified roles in your VO as: *roots *user support *users who compute their jobs utilizing cluster manwe and one storage element. So create three groups (roots, support, users) and assign resources to each group according to specific requirements for the roles. Each group has assigned a complex set of resources. *Group "roots" have assigned root access to resources (manwe and storage). *Group "support" have assigned both root access and computing access to cluster and storage. *Group "users" have assinged computing access to the computational resources (manwe and storage element). Main advantage is when new member (new manager/support/user) enters into system, you can simply add it to one group according to his role where all necessary resources and access rights are already assigned. It is easy, saves time and you are sure that new member has all resources he needs from the beginning. '''Bad example''': You create many single-purpose groups, for example: *Group to root access to the cluster manwe, whose members have only root access to cluster manwe. *Group to root access to storage element, whose members have only root access to storage element. *Group to computing access to the cluster manwe, whose members have only computing access to cluster manwe. *Group to computing access to storage element, whose members have only computing access to storage element. and assign members to each group separately. In your case it means assign all managers (one by one) as members of two groups with root access, all user supports as members of all groups and all users as members of computing groups. As the result, when you want to add new member (new manager/support/user), you need to add him to many groups instead of one as in the previous case. Our example is very small, but when managing bigger VO, it is easy to forget to somebody. }} <hr> ==== Tutorial to assign group to resource==== [[My role is VO MANAGER|''VO MANAGER]] ''role needed'' * visit {{PerunGUILink|rel=vo/list perunGUI VOs pages}} * use '''Select VO''' [[File:Building.png|frameless|alt=VO|]] from the left menu to display list of VOs which you can manage * click on the row with VO name to select VO which you want to work with * there are two ways how to assign group to resource ** click on the button '''Groups''' at upper bar OR click on the '''Groups''' [[File:group.png|frameless|alt=VO|]] option at the left menu ** click on the row with name of required group and detail of this group is displayed ** press the button '''Resources''' at upper bar OR click on the '''Resources''' [[File:server_group.png|frameless|alt=VO|]] option at the left menu ** click on the button '''Add...''' [[File:add.png|frameless|alt=VO|]] and list of resources for adding is displayed ** select required resource (or resources) by ticking check-box, press the button '''Add''' [[File:add.png|frameless|alt=VO|]] and required group is assigned with required resource ** ...................OR..................... ** click on the button '''Resources''' at upper bar OR click on the '''Resources''' [[File:server_group.png|frameless|alt=VO|]] option at the left menu ** click on the row with selected resource ** click on the button '''Assigned groups''' and list of recently assigned groups is displayed ** press the button '''Add...''' [[File:add.png|frameless|alt=VO|]] to display list of groups offered to assign ** select required group (or groups) by ticking check-box, press the button '''Add''' [[File:add.png|frameless|alt=VO|]] and required group is assigned with required resource ** you can use button [[File:Filter.png|frameless|alt=VO|]] '''Filter''' to find group you want to add ===== Assign group to resource by using tags ===== [[My role is VO MANAGER|''VO MANAGER]] ''role needed'' * visit {{PerunGUILink|rel=vo/list perunGUI VOs pages}} * use '''Select VO''' [[File:Building.png|frameless|alt=VO|]] from the left menu to display list of VOs which you can manage * click on the row with VO name to select VO which you want to work with * click on the button '''Groups''' at upper bar OR click on the '''Groups''' [[File:group.png|frameless|alt=VO|]] option at the left menu * click on the row with name of required group and detail of this group is displayed * press the button '''Resources''' at upper bar OR click on the '''Resources''' [[File:server_group.png|frameless|alt=VO|]] option at the left menu * click on the button '''Add...''' [[File:add.png|frameless|alt=VO|]] and list of resources for adding is displayed * insert name of tag which you want to use into filter area and click on the button '''Filter''' [[File:Filter.png|frameless|alt=VO|]] - tagged resources only are listed (currently only ''computational node'' and ''storage'' tags are available) * by ticking upper check-box mark all listed resources, press the button '''Add''' [[File:add.png|frameless|alt=VO|]] and required group is assigned with all tagged resources * list of assigned resources is dispayed under selection area * click on [[File:cross.png|frameless|alt=VO|]] '''Close''' button to finish adding of resources [[Category:Perun]] in1aupgy7jlq2kq449225nbsjlbw4tx 10 403 2103 917 2016-05-12T14:21:00Z Root 1 wikitext text/x-wiki '''Notice:''' This application supports parallel computing (MPI, OpenMP) which can have weird consequences. For more details about parallel computing visit the page [[How to compute/Parallelization]].<br /> [[Category:MPI (en)]] g62i70riteliczvo19vy5a4kyy0yhkv 0 527 2090 1951 2016-05-12T07:29:52Z Root 1 Root přesunul stránku [[How to compute/Parallel jobs]] na [[How to compute/Parallelization]] bez založení přesměrování: Nahrazení textu „How to compute/Parallel jobs“ textem „How to compute/Parallelization“ wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[:Category:MPI_(en)|Applications with MPI]]|}} {{BoxPoložka|[[How to compute/Requesting resources|Requesting resources]]|}} }} ''Parallel computing'' can '''significantly shorten time''' of your job because the job uses multiple resources at once. You can read more about parallel computing [https://en.wikipedia.org/wiki/Message_Passing_Interface here]. MetaCentrum offers two ways of parallel computing - [https://en.wikipedia.org/wiki/OpenMP OpenMP] and [https://en.wikipedia.org/wiki/Message_Passing_Interface MPI], or you can combine both. ==OpenMP== {{RedInfo|Make sure the '''OMP_NUM_THREADS''' environment variable is set before running your job}} If your application is able to use multiple threads via a shared memory, '''[[How to compute/Requesting resources|request]] a single node with multiple processors'''. For example: [[Image:Parallel1.png]] Make sure that before running your application, the ''OMP_NUM_THREADS'' environment variable is appropriately set. Otherwise your application will use all the available cores on the node and influence other jobs. ===Setting OMP_NUM_THREADS variable=== Type the following line to you batch script. *TOM(interaktivní před nebo po ?). * <source lang="bash">export OMP_NUM_THREADS=$PBS_NUM_PPN #restricts count of parallel processes to number of processors given by scheduling system</source> ==MPI== {{YellowInfo|Running an MPI computation is possible via '''mpirun''' command}} If your application consists of multiple processes communicating via a message passing interface (see [[:Category:MPI (en)|category applications with MPI]]), '''[[How to compute/Requesting resources|request]] for a set of nodes''' (with arbitrary number of processors). For example: [[Image:NonInfinibandExample.png]] * Make sure the appropriate openmpi/mpich2/mpich3/lam module is loaded into your environment before running your computation ===Speed acceleration=== You can request special nodes, which are interconnected by a low-latency [https://en.wikipedia.org/wiki/InfiniBand '''InfiniBand connection'''] to accelerate the speed of your job. For example: [[Image:infinibandExample.png]] * The InfiniBand is automatically detected when running an MPI computation using InfiniBand. You can begin your computation in the common way. For example: <source lang="bash"> mpirun myMPIapp </source> ==MPI and OpenMP interaction== If your application supports both types of parallelization (MPI and OpenMP), you can ''interact'' them. You have to run your computation '''correctly''', otherwise the job might get to conflict with the [[:Category:Scheduling system|scheduling system]], which will kill your job. Examples of correct using the OpenMP library: {{YellowInfo|First two examples are interchangeable, however they can influence the calculation speed. Try both and select the faster method}} {| class="tabulka" ! style="background-color:LightSeaGreen;"|Requested resources !! style="background-color:LightSeaGreen;"|Example |- | 1 device, more processors | <source lang="bash">export OMP_NUM_THREADS=$PBS_NUM_PPN mpirun -n 1 /path/to/program ...</source> |- | 1 device, more processors | <source lang="bash">export OMP_NUM_THREADS=1 mpirun /path/to/program ...</source> |- | 2 device, more processors | <source lang="bash">cat $PBS_NODEFILE |uniq >nodes.txt export OMP_NUM_THREADS=$PBS_NUM_PPN mpirun -n 2 --hostfile nodes.txt /path/to/program ...</source> |} [[Category:Grid services]] [[Category:MPI (en)]] 6a7020y4z5yl7xwxtlfbz973gf6nki3 0 470 2107 2106 2016-05-12T15:43:16Z Root 1 wikitext text/x-wiki ([[Čelní uzel|Czech version]]) {{Box|Related topics {{BoxPoložka|[[How to compute/Accessing machines|Accessing machines]]|}} }} '''Frontend''' is a machine where you can log in directly, without reservation. It is mainly the '''entry point''' to MetaCentrum machines. Frontends are the following machines: {| class="tabulka" ! style="background-color:LightSeaGreen;"|Machine name(Frontend) ! style="background-color:LightSeaGreen;width=70px;"|System ! style="background-color:LightSeaGreen;"|Home directory ! style="background-color:LightSeaGreen;width=350px;"|PBS server |- | skirit.ics.muni.cz | bgcolor=#c9ffc9|Debian 7 | /storage/brno2/home/ | (PBS server @arien) |- | tarkil.cesnet.cz | bgcolor=#c9ffc9|Debian 7 | /storage/praha1/home/ | (PBS server @arien) |- | nympha.zcu.cz | bgcolor=#c9ffc9|Debian 7 | /storage/plzen1/home/ |(PBS server @arien) |- | hermes.metacentrum.cz | bgcolor=#c9ffc9|Debian 7 | /storage/budejovice1/home/ | (PBS server @arien) |- | minos.zcu.cz | bgcolor=#c9ffc9|Debian 7 | /storage/plzen1/home/ | (PBS server @arien) |- | perian.ncbr.muni.cz | bgcolor=#c9ffc9|Debian 7 | /storage/brno2/home/ | (PBS server @arien) |- | onyx.ncbr.muni.cz | bgcolor=#c9ffc9|Debian 7 | /storage/brno2/home/ | (PBS server @arien) |- |zuphux.cerit-sc.cz | bgcolor=#c9ffc9|Debian 8 | /storage/brno3-cerit/home/ |(PBS server @wagap [http://www.cerit-sc.cz/cs/docs/quickstart/ návod]) |- |} The same names are also aliased to the '''metacentrum.cz''' domain, e.g. skirit.metacentrum.cz. If you are unsure about choosing the right machine, you can try the random solution generated by this bash script :-) F=(skirit tarkil nympha hermes minos perian onyx zuphux); ssh ${F[$[$RANDOM%${#F[@]}]]}.metacentrum.cz == Usage of frontends == Please consider that frontends are shared among all users. If you need to run interactively some demanding activity, like working with huge amount of data or softare compilation, please don't use the frontend but rather reserve some computing node for an [[How to compute/Interactive jobs|interactive jobs]]. Such operations could slow down the frontend and bother other users. {{RedInfo|Use the [[Working with data/Direct access to data storages|direct access to data storages]] to manipulate with '''huge amounts of data'''.}} [[Category:Grid services]] [[Category:Infrastructure]] hp3jhcbkryib25urp7r7ykph882smul 0 543 2088 1978 2016-05-12T07:29:31Z Root 1 Nahrazení textu „How to compute/Parallel jobs“ textem „How to compute/Parallelization“ wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[How to compute]]|}} {{BoxPoložka|[[Types of MetaCentrum machines]]|}} }} This page provides information about ''grids'' and ''supercomputers'' - technology used in MetaCentrum . You will learn which problems can be solved with the help of MetaCentrum resources, what software you need, how to acces MetaCentrum and how to prepare and submit your first job. ==Grids== The Grid term was introduced in 1998 by [http://en.wikipedia.org/wiki/Carl_Kesselman Carl Kesselman] and [http://en.wikipedia.org/wiki/Ian_Foster Ian Foster] in book [http://www.globus.org/alliance/publications/papers.php#chapter2 The Grid: Blueprint for a New Computing Infrastructure]. Though this word has many meanings, it requires some explanation here. === Evolving of definition === Carl Kesselman and Ian Foster define grid as follows: {{GreenBox|'''1998/99 A computational grid '''is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities. Grid has to fulfill following three points: * coordinates resources that are not under centralized management * use standard, open, generic protocols and interfaces * provides non-trivial quality of services (more than each individual part of its own) '''2001''' ... coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations. '''2002''' ...a Grid is a system that: coordinates resources that are not subject to centralized control … using standard, open, general-purpose protocols and interfaces … to deliver nontrivial qualities of service …}} Computational grid is described by analogy with electrical power network - around year 1910 every electrified building had its own electricity generator but those generators were not interconnected. There was no possibility to use larger capacity than capacity of a single generator, total capacity of all generators were not used effectively and establishment of electricity was expensive. The real utilization of electricity usage begun after start of interconnected large power stations and distribution network to the consumers. This allowed cheap, ubiquitous and standardized source of electric power. Similarly, every organization today manages its own computational capacities (computers, disc capacity, software, data, specialized hardware) that can not be effectively shared with others. Computational grid should enable such effective sharing of computational capacities among organizations. ===Features of Grid=== * Heterogeneous hardware and software * Loosely-coupled machines * Potentially unreliable low-speed network * Several distributed locations * Several administrative domains * Potentially insecure * Grid middleware (ARC, gLite, etc.) * Best for embarrassingly parallel jobs * Focus on high throughput / capacity computing ====Scientific Areas of Interest==== Mostly Computational Sciences * Computer Science * Computational Chemistry, Cheminformatics * Bioinformatics, Biomedical Computing (e.g., Imaging) * Physics (High Energy Physics, Plasma Physics, Solid State Physics, Theoretical Physics, QCD, etc.) * Earth Observation, Satellite Imaging, Meteorology, Climatology, Geography, Geology, etc. * Nanotechnology * Many more ==Supercomputers== A supercomputer is a computer with a high-level computational capacity compared to a general-purpose computer. ===What are the supercomputers good for=== Supercomputers are ordinary computers, just more powerful than "normal" computers. One has to realise why they are more powerful. In one type of supercomputers &ndash; considered as the real supercomputers by public &ndash; it is because their processors are much faster in computing some exact operations. In particular this is the case of vector supercomputers that are able to process whole vectors (n-tuples) of numbers at a speed of normal processors working only with scalar operands (separate numbers). Nevertheless we don't have a vector supercomputer in MetaCentrum yet. Generally, the supercomputers needn't have faster processors than are available in PC's or simple servers, but they have a lot of these processors. That has a fundamental impact on which problems supercomputers can compute faster and which they can't. ===If one labourer does a job in 2 hours, then 100 labourers...=== Real Life example: Try to imagine a large corn field and one farmer with a scythe. It will take two weeks to cut the field. When you send two farmers to cut the field, it will take one week. 14 farmers cut the field for one day. 140 farmers cut the field in an hour. 8400 farmers cut the field in a minute. It´s possible to increase of number of farmers, so maybe 8400 farmer cut the field in a minute. Cutting field is a good example of parallelizable job, more used workers means the task is done faster. But try to imagine one broken watch. One watchmaker repairs it for 10 hours. Two watchmakers repair it for 10 hours too as well as 8000 watchmakers, because only one watchmaker can repair the watch and rest of watchmakers have to wait. Repairing the watch is non-parallelizable job. The supercomputer isn´t one worker (= processor), who works faster then another workers. But supercomputers are a big group well coordinated workers. It depends only on you if they work hard or if the most of them wait idly. It is worthwhile to use supercomputers for computational jobs solved by co-work of many processors. If each step of your job depends on the result of previous step, it´s not possible to use more than one processor. So supercomputers can´t accelerate solution of your jobs. If it´s possible to separate you computational jobs into parallel processed parts, welcome in MetaCentrum. ==Long-running jobs == MetaCentrum can offer you environment for long-running jobs, even if you don´t have the parallellizable job. Some jobs could take months, so during this time black out or collapse of the operating system could happen or simply somebody turns off your PC. MetaCentrum has machines running for a long time. Their power supply is backed up by accumulators and the diesel generator. They are placed in guarded air-conditioned hall. If you have computational jobs, possibly running for months, welcome on MetaCentrum too! ==Clusters and Grids== Let me introduce you supercomputers´s relatives – clusters and grids, which are also used in MetaCentrum. Cluster is same computers connected together by communication network. Grid is many computers belonged to various organizations free connected by net. Grid´s computers can be various. Clusters and grids provide computational power on the same principle like supercomputers. They also offer many processors. They differ in their interconnection. In the supercomputer there are processors connected by very fast internal network with minimal level of delay. These processors have access to shared memory. In clusters and grids we use common Internet, IP protocol and non-shared memory. The advantage of clusters and grids is lower price, so it´s possible to acquire more processors for same price. In clusters you can link processors to each other (by suitable options, switched nets) or we can use special high-speed nets with minimal level of delay (myrinet or Infiniband). In Grids we are dependent on large nets, of which total permeability is significantly smaller and the level of delay is many times higher. [[How to compute/Parallelization|More info...]] The increasing level of delay affects parallellizable jobs. Let me remind you example with farms and the corn field. We didn´t count with time necessary for organization their work. So it could happened very easy that preparation of the job can take more time then job´s solution. Parallel computers prepare their jobs faster than clusters and clusters faster then Grids. Jobs can exchange messages faster and more often in supercomputer. Despite this fact clusters and Grids are effective option (economical and productive) of specialized supercomputers. It´s possible to use them for solution of large parallellizable jobs. We offer you supercomputers with many processors and shared memory as well as clusters composed of hundreds machines with dual or quad-core processors. We also provide access to international Grids, where a large amount of processors are available. Don´t forget: large amount available processors means increase in difficulty of working with them. Don´t worry we are able to help you. You can start with dual-core processors and then go to more advanced configurations. The current list of available machines on web [https://www.metacentrum.cz/en/resources/hardware.html List of machines in MetaCentrum] If you are interested in using our services please continue to [[How to compute]] [[Category:Infrastructure]] e7c9jjetzuzgwe1jyly0u5ul6haqsiw 0 544 2007 2006 2016-05-05T12:11:16Z Root 1 wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[http://www.metacentrum.cz/en/resources/hardware/index.html List of available hardware]|}} }} According to login behavior you can meet with four types of machines in MetaCentrum. These are frontends and computing machines, and virtual machines and their controllers. This page describes the ability of user to login on each type of machine. ==Virtual machines== Due to the effectivity reasons are some machines in MetaCentrum virtual. This means that there is one physical machine which runs the [http://en.wikipedia.org/wiki/Xen XEN software] to become several virtual machines. Eg. one machine runs several operating systems which might be even different. We use this technology mainly for operation of frontends (described later) and also for [[:Category:Clouds|Cloud]] operation. Physical machine which contains several virtual machines is called '''dom0''' and the virtual machine is '''domU'''. From the perspective of possibility of user login on the machines you: * '''cannot''' login to dom0 machine, * domU machine is fully accessible (with root privileges) when it's a part of cloud, otherwise while used as a frontend, you have the common user rights. ==Frontends and working nodes== ''Frontends'' are used for setting up of job and for receiving of results or any other interactive work. These are the machines which you can log in anytime. Their list is available [[Frontend|here]]. The most of the frontends are virtual machines. Topical guide [[How to compute]] provides information about using frontends for computing. ''Working nodes'' are machines which physically count the user jobs. So they are mainly used for non-interactive work which is assigned to them by scheduling system. You can login in them only with a good reason, for example for cleaning up a mess after failed job or for downloading the data after job failure. Imagine that the working node is in time of your login used by someone's else job which you will slow down by your activity. [[Category:Infrastructure]] jv8mq7di51ry2k3msenxooko3apipj2 0 65 229 228 2016-01-10T13:24:23Z Root 1 naimportována 1 revize wikitext text/x-wiki = Úvod = Toto je detailní dokumentace k Hadoop clusteru v MetaCentru. Základní informace o Hadoopu a přístupu k Hadoop clusteru viz [https://wiki.metacentrum.cz/wiki/Hadoop Hadoop]. Úvodní stránka s odkazy na dokumentaci k Hadoopu viz [https://wiki.metacentrum.cz/wiki/Kategorie:Hadoop Kategorie:Hadoop]. = Detaily = {| class="wikitable" ! style="text-align:left;"|Subsystém ! ! style="text-align:left;"|Komponenta ! style="text-align:left;"|Hostname ! style="text-align:left;"|Popis |- | HDFS || style="text-align:right;"| 2⨯ || '''namenode''' || <tt>hador-c1.ics.muni.cz</tt>, <tt>hador-c2.ics.muni.cz</tt> || Hadoop file system metadata |- | HDFS || style="text-align:right;"| 24⨯ || '''datanode''' || <tt>hador1.ics.muni.cz</tt> ... <tt>hador24.ics.muni.cz</tt> || Hadoop file system data nodes |- | HDFS || style="text-align:right;"| 3⨯ || '''journalnode''' || <tt>hador-c1.ics.muni.cz</tt>, <tt>hador-c2.ics.muni.cz</tt>, <tt>hador.ics.muni.cz</tt> || Hadoop file system HA journal nodes |- | YARN || style="text-align:right;"| 2⨯ || '''resourcemanager''' || <tt>hador-c1.ics.muni.cz</tt>, <tt>hador-c2.ics.muni.cz</tt> || YARN resource control |- | YARN || style="text-align:right;"| 24⨯ || '''nodemanager''' || <tt>hador1.ics.muni.cz</tt> ... <tt>hador24.ics.muni.cz</tt> || YARN computing nodes |- | MAPRED || style="text-align:right;"| 1⨯ || '''historyserver''' || <tt>hador-c2.ics.muni.cz</tt> || Map/Reduce jobs history server |- | ZOOKEEPER || style="text-align:right;"| 3⨯ || '''zookeeper''' || <tt>hador-c1.ics.muni.cz</tt>, <tt>hador-c2.ics.muni.cz</tt>, <tt>hador.ics.muni.cz</tt> || Zookeeper servers (HA, sync, ...) |- | || style="text-align:right;"| 1⨯ || '''frontend''' || <tt>hador.ics.muni.cz</tt> || Hadoop client |} = Úložiště dat = '''1) řídicí servery a frontend''' * OS na RAID 1 (mirror) [2 x 1 TB] * data na RAID 6, 5.5 GB [8 x 1 TB] ** hador-c1, hador-c2 (HDFS): metadata HDFS clusteru ** hador (frontend): scratch pro uživatele Ukládání kopie metadat na vzdálené NFS není použito (==&gt; nižší ochrana dat, ale vyšší spolehlivost clusteru). '''2) datové uzly''' * OS na RAID 1 (mirror) [2 x 1 TB] * data na fyzických discích přímo [12 x 4 TB] * scratch z částí datových disků (stripe raid) [12 x 70 GB] Redundance řešena na úrovni HDFS, replikační faktor nastaven na 4. = Přístup k datům = Cluster je propojen přes rychlejší síť infiniband (s IP adresami z lokálního rozsahu) a zárověň je přístupný i zvnějšku přes klasickou síť. Propojení infinibandem je také mezi frontendem hador.ics.muni.cz a zbytkem clusteru. Zároveň je ve stejném clusteru připojen i storage brno6. Pro rychlejší přesuny dat mezi HDFS a lokálním diskem lze využít: '''1) lokální adresář /scratch na hador.ics.muni.cz''': hdfs dfs -put /scratch/USER/books . hdfs dfs -get books /scratch/USER/books2 '''2) storage brno6''': hdfs dfs -put /auto/brno6/home/USER/books . hdfs dfs -get books /auto/brno6/home/USER/books2 HDFS je přístupné také jako připojený adresář, ale přesun dat je velmi pomalý. Vhodné pro jiné operace: '''3) HDFS-NFS brána na hador.ics.muni.cz''': find /hdfs/user/`id -un` -type f = Administrátorská dokumentace = '''instalace''': [https://wiki.metacentrum.cz/metawiki/U%C5%BEivatel:Valtri/Hadoop/Installation Uživatel:Valtri/Hadoop/Installation] '''changelog''': [https://wiki.metacentrum.cz/metawiki/U%C5%BEivatel:Valtri/Hadoop/LogBook Uživatel:Valtri/Hadoop/LogBook] [[Kategorie:Česky]] [[Kategorie:Hadoop]] 140nqcp9j1xdrd1deis327e32u59scf 0 66 233 232 2016-01-10T13:24:23Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Category:Hadoop]] [[Category:Česky]] Hadoop je určen pro všechny uživatele MetaCentra. Před začátkem používání je třeba podat přihlášku do skupiny '''hadoop''', odkaz je na stránce http://www.metacentrum.cz/cs/hadoop/ fyyb7ubflfpwqowlqiunac1k462yres 0 67 235 234 2016-01-10T13:24:24Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Category:Hadoop]] [[Category:Česky]] == Úvod == Tato dokumentace se týká testovací instance Hadoop clusteru. Plná dokumentace k produkčnímu Hadoop clusteru se nachází na: [[Hadoop]]. == Základní informace == * frontend: '''took10.ics.muni.cz''' ** HDFS NFS Gateway: adresář '''/hdfs''' * HDFS Name Node (HA): took25.ics.muni.cz, took35.ics.muni.cz * YARN Resource Manager (HA): took25.ics.muni.cz, took35.ics.muni.cz * MapReduce History Server: took35.ics.muni.cz * uzly: ** took43.ics.muni.cz ** took44.ics.muni.cz ** took45.ics.muni.cz ** took46.ics.muni.cz == Data == * replikační faktor: 2 * data: 4 ⨯ 150 GB (<span style="font-size: bigger">⇒</span> '''300 GB''') * cache: 4 ⨯ 20 GB * metadata: 2 ⨯ 10 GB == Instalovaný SW == '''[[Hadoop_Hadoop|Hadoop]]''' 2.6.0 - distributed storage and processing of very large data sets '''[[Hadoop_HBase|HBase]]''' 1.0.0 - distributed, scalable, big data store '''[[Hadoop_Hive|Hive]]''' 1.1.0 - data warehouse software facilitates '''[[Hadoop_Pig|Pig]]''' 0.12.0 - platform for analyzing large data sets '''[[Hadoop_Spark|Spark]]''' 1.3.0 - fast and general engine for large-scale data processing Používá se Cloudera 5.5.0. == Kterak se přihlásit == Frontend je na '''took10.ics.muni.cz''': kinit ssh -K took10.ics.muni.cz Více informací viz [[Hadoop#Kterak_se_přihlásit]]. == Webový přístup == Testovací certifikační autorita: * URL: https://scientific.zcu.cz/test-CA/ * certifikát: [https://scientific.zcu.cz/test-CA/Monsters-Inc-Root-CA.pem Monsters-Inc-Root-CA.pem] * SHA-256: 83:3D:A7:52:7F:2A:65:B4:AB:ED:91:5B:C3:9F:6A:EE:7F:6C:C1:5E:25:4F:4A:97:06:DE:40:59:DF:FB:F5:F6 * SHA-1: A9:E0:CB:55:61:7E:CC:29:2C:03:94:D2:1B:99:24:C6:13:79:B8:BD Servisní stránky: * HDFS: [https://took25.ics.muni.cz:50470 https://took25.ics.muni.cz:50470], [https://took35.ics.muni.cz:50470 https://took35.ics.muni.cz:50470] * YARN: [https://took25.ics.muni.cz:8090 https://took25.ics.muni.cz:8090], [https://took35.ics.muni.cz:8090 https://took35.ics.muni.cz:8090] * MapRed History Server: [https://took35.ics.muni.cz:19890 https://took35.ics.muni.cz:19890] * HBase: [https://took25.ics.muni.cz:60010 https://took25.ics.muni.cz:60010] * Spark: [http://took35.ics.muni.cz:18082 http://took35.ics.muni.cz:18082] s9n3aoziz8b7xna7d74ipwzfyeszr78 0 68 237 236 2016-01-10T13:24:24Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Category:Hadoop]] = HBase = == Informace == Tabulky potřeba vyrábět v namespacu u každého uživatele, např.: create 'USER:TABLE', 'CF' Vyrobit prázdnou tabulku lze i bez namespacu nebo v cizím namespacu, ale těmto tabulkám budou chybět potřebná práva. == Připojení == === Shell === Na frontendu (viz [[Hadoop#Základní informace]]), přístup přes Kerberos (nahraďte '''USER''' za skutečné uživatelské jméno): hbase shell create 'USER:test', 'cf' put 'USER:test', 'row1', 'cf:a', 'value1' put 'USER:test', 'row2', 'cf:a', 'value2' put 'USER:test', 'row3', 'cf:a', 'value3' put 'USER:test', 'row4', 'cf:a', {'a'=>1, 'b'=>2} scan 'USER:test' get 'USER:test', 'row1' describe 'USER:test' disable 'USER:test' drop 'USER:test' === RPC API === * lze využít [https://github.com/OpenTSDB/asynchbase asynchbase] * vyžadována verze >= 1.7snap (k datu 2015-07-20 branch '''next''') * nutno nastavit property ''java.security.auth.login.config=jaas.conf'' '''asynchbase.properties''': hbase.zookeeper.quorum = hador-c1.ics.muni.cz:2181,hador-c2.ics.muni.cz:2181,hador.ics.muni.cz:2181 hbase.security.auth.enable = true hbase.security.authentication = kerberos hbase.kerberos.regionserver.principal = hbase/_HOST@ICS.MUNI.CZ hbase.sasl.clientconfig = Client '''jaas.conf''' (nahraďte ''hawking'' za skutečný login name): Client { com.sun.security.auth.module.Krb5LoginModule required useKeyTab=false useTicketCache=true principal="hawking@META"; }; '''Example.java''' (nahraďte ''hawking:test'' a ''row1'' za skutečné hodnoty): import java.util.ArrayList; import org.hbase.async.Config; import org.hbase.async.HBaseClient; import org.hbase.async.GetRequest; import org.hbase.async.KeyValue; final class Example { static final String configFile = "asynchbase.properties"; static final String table = "hawking:test"; static final String key = "row1"; public static void main(final String[] args) throws Exception { Config config = new Config(configFile); HBaseClient client = new HBaseClient(config); GetRequest get = new GetRequest(table, key); try { ArrayList<KeyValue> result = client.get(get).joinUninterruptibly(); System.out.println("Get result=" + result); } catch (Exception e) { System.out.println("Get failed:"); e.printStackTrace(); } client.shutdown(); } } '''Makefile''': CP=$(shell ls -1 asynchbase-*.jar async-*.jar netty-*.jar slf4j-*.jar protobuf-java-*.jar zookeeper-*.jar log4j-*.jar | tr '\n' ':') all: Example.java javac -cp $(CP) Example.java run: java -cp $(CP):. -Djava.security.auth.login.config=$(shell pwd)/jaas.conf Example '''pom.xml''' pro Maven (<tt>Example.java</tt> uložit do <tt>src/main/java/cz/cesnet/hadoop</tt>): <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>cz.cesnet.hadoop</groupId> <artifactId>hbase-example</artifactId> <version>0.0.1</version> <name>example</name> <properties> <maven.compiler.target>1.7</maven.compiler.target> <maven.compiler.source>1.7</maven.compiler.source> </properties> <repositories> <repository> <id>sonatype-staging</id> <url>https://oss.sonatype.org/content/repositories/staging/</url> </repository> </repositories> <dependencies> <dependency> <groupId>org.hbase</groupId> <artifactId>asynchbase</artifactId> <version>1.7.0-SNAPSHOT</version> </dependency> </dependencies> </project> == Servisní stránky == '''Cluster''': * https://hador-c1.ics.muni.cz:60010 - stav HBase (master) '''Jednotlivé uzly''' (X=1..24): * https://hadorX.ics.muni.cz:60030/ - HBase regionserver 3a99yfs0jhxyr1eya8bmvdhmza0vgwu 0 69 239 238 2016-01-10T13:24:24Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Category:Hadoop]] = Úvod = Tato dokumentace popisuje, jak nastavit lokální Hadoop frontend. Konfigurce lze získat z frontendu Hadoopu v MetaCentru: '''hador.ics.muni.cz'''. Jsou generovány automaticky nástrojem puppet, a tak se mohou čas od času měnit. = Systém = == Kerberos klient == Je potřeba správně fungující kerberos klient pro realm META a ICS.MUNI.CZ, tj.: # '''<tt>/etc/krb5.conf</tt>''' (např. z '''hador.ics.muni.cz''', viz také [[Kerberos na lokální stanici (linux)]]) # správný lokální čas (doporučeno použít '''ntp''') == Repozitář == Doporučujeme instalovat totožné verze SW (tj. použít distribuci Cloudera a lokální mirror s odzkoušenými verzemi používané pro MetaCentrum). Je možno použít i originální repozitář Coudery (http://archive.cloudera.com/cdh5/). V případě použití platformy Debian 7/wheezy: echo 'deb [arch=amd64] http://scientific.zcu.cz/repos/hadoop/cdh5/debian/wheezy/amd64/cdh wheezy-cdh5 contrib' > /etc/apt/sources.list.d/cloudera.list apt-key adv --fetch-key http://scientific.zcu.cz/repos/hadoop/archive.key apt-get update == Java == Podporované verze Javy pro CDH5 jsou: openjdk 7, openjdk 8. Instalace pro Debian 7/weeezy: apt-get install openjdk-7-jre-headless == Hadoop == * instalace: apt-get install hadoop-client * zkopírovat konfigurace z frontendu: /etc/hadoop/conf/core-site.xml /etc/hadoop/conf/hdfs-site.xml /etc/hadoop/conf/yarn-site.xml /etc/hadoop/conf/mapred-site.xml == Hive == * instalace: apt-get install hive * zkopírovat konfigurace z frontendu: /etc/hive/conf/hive-site.xml == HBase == * installace: apt-get install hbase mkdir -p /var/lib/hbase/local/jars chown -R hbase:hbase /var/lib/hbase/local * zkopírovat konfigurace z frontendu: /etc/hbase/conf/hbase-site.xml == Spark == Potřeba nastavit také [[#Hadoop]], protože se v MetaCentru používá v režimu s YARNem. * instalace: apt-get install spark-python * zkopírovat konfigurace z frontendu: /etc/spark/conf/spark-defaults.conf /etc/profile.d/hadoop-spark.csh /etc/profile.d/hadoop-spark.sh Místo souborů v <tt>/etc/profile.d/</tt> je možno nastavit proměnné v <tt>~/.bashrc</tt>: export HADOOP_CONF_DIR=/etc/hadoop/conf export YARN_CONF_DIR=/etc/hadoop/conf == Pig == * instalace: apt-get install pig #volitelně: apt-get install pig-udf-datafu * zkopírovat konfigurace z frontendu: /etc/profile.d/hadoop-pig.sh /etc/profile.d/hadoop-pig.csh Místo souborů v <tt>/etc/profile.d/</tt> je možno nastavit proměnnou v <tt>~/.bashrc</tt>: export HADOOP_MAPRED_HOME=/usr/lib/hadoop-mapreduce 7j85c7ys2it0u0mbgv127umkhyddki0 0 64 259 231 2016-01-21T20:52:04Z Root 1 wikitext text/x-wiki [[Category:Hadoop]] = Základní informace = * frontend: '''hador.ics.muni.cz''' * HDFS namenode: hador-c1.ics.muni.cz * YARN Resource Manager, MapRedure History Server: hador-c2.ics.muni.cz * uzly: hador1.ics.muni.cz - hador24.ics.muni.cz == Dostupné hardwarové prostředky == * '''CPU''': 24(+3) strojů x 16 jader x 2 hyperthreading (Intel® Xeon® CPU E5-2630 v3 @ 2.40GHz) * '''RAM''': 24(+3) strojů x 128 GB * '''data''': 1.02 PB úložiště dat (4 replikace x 261 TB) * '''režie a metadata''': 78 TB (5 TB HDFS metadata, 54 TB OS, 4 TB scratch, raid overhead, ...) == Instalovaný SW == '''[[Data_v_Hadoopu|Hadoop]]''' 2.6.0 - distributed storage and processing of very large data sets '''[[Hadoop_HBase|HBase]]''' 1.0.0 - distributed, scalable, big data store '''[[#Hive|Hive]]''' 1.1.0 - data warehouse software facilitates '''[[#Pig|Pig]]''' 0.12.0 - platform for analyzing large data sets '''[[#Spark|Spark]]''' 1.3.0 - fast and general engine for large-scale data processing Používá se Cloudera 5.4.7. == Kterak se přihlásit == Frontend je na '''hador.ics.muni.cz''': kinit ssh -K hador.ics.muni.cz (volba <tt>-K</tt> forwarduje existující Kerberos lístek) === Dlouhé úlohy === Pro delší úlohy je nutno ještě inicializovat lístek s delším '''renewable time''' (po přihlášení na hador.ics.muni.cz): kinit -r 7d === Úloha na pozadí === Při odhlášení se Kerberos lístky mažou. Pokud úloha běží na pozadí a má běžet i po odhlášení, je potřeba si lístky zkopírovat a nastavit <tt>KRB5CCNAME</tt>. Lze využít wrapper <tt>/usr/local/bin/launch</tt>, který se o vše postará: kinit -r 7d nohup launch program ... Nebo lze pracovat s lístky přímo: hawking@hador:~$ klist Credentials cache: FILE:/tmp/krb5cc_60303_Iw6Y1NTTCf Principal: hawking@META hawking@hador:~$ cp /tmp/krb5cc_60303_Iw6Y1NTTCf /tmp/krb5cc_`id -u`_long hawking@hador:~$ export KRB5CCNAME=FILE:/tmp/krb5cc_`id -u`_long hawking@hador:~$ nohup program ... Po skončení je vhodné provést <tt>kdestroy</tt> (se stejným <tt>KRB5CCNAME</tt>): hawking@hador:~$ export KRB5CCNAME=FILE:/tmp/krb5cc_`id -u`_long hawking@hador:~$ kdestroy === Vzdálený klient === Popis konfigurace vlastního klienta viz [[Hadoop Klient]]. = Hadoop = == Informace == * max. délka úlohy omezena: ** max. refresh time uživatelského '''Kerberos ticketu''' ** max. lifetime uživatelských tokenů Hadoopu, '''7 dní''' * zabezpečení dat: řešen přístup k datům, konzistence dat, ale lokálně se nešifruje == Příklady == Na frontendu (viz [[#Základní informace]]), přístup přes Kerberos: hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar pi 2 10 hdfs dfs -put /etc/hadoop/conf input hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar grep input output 'dfs[a-z.]+' hdfs dfs -cat output/* hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar randomwriter out/ hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar teragen 100 gendata hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar terasort gendata sorted hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar teravalidate gendata reportdata == Získání logů == yarn application -list yarn logs -applicationId APPLICATION_ID == Webový přístup == Přes web lze přistupovak k datům ([http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/WebHDFS.html WebHDFS], read-only) nebo jsou k dispozici servisní informace clusteru. Pro webový přístup k Hadoop clusteru je nutný protokol SPNEGO (Kerberos přes HTTP(S)). === Prohlížeč === Postup povolení protokolu SPNEGO v prohlížeči: * firefox [http://www.microhowto.info/howto/configure_firefox_to_authenticate_using_spnego_and_kerberos.html]: *# <tt>about:config</tt> *# nastavit '''network.negotiate-auth.trusted-uris=ics.muni.cz''' (nebo čárkami oddělené hostnamy serverů) *# pro Windows: *## nainstalovat [https://wiki.metacentrum.cz/wiki/Pou%C5%BEit%C3%AD_programu_Kerberos_for_Windows Kerberos] *## nastavit také '''network.auth.use-sspi=false''' * chrome: [http://www.microhowto.info/howto/configure_chromium_to_authenticate_using_spnego_and_kerberos.html] ** parametr '''--auth-server-whitelist="hador*.ics.muni.cz"''' ** trvalé nastavení: vyrobit soubor '''/etc/opt/chrome/policies/managed/hadoop-metacentrum.json''':<br/><tt>{<br/>&nbsp;&nbsp;"AuthServerWhitelist": "hador*.ics.muni.cz",<br/>&nbsp;&nbsp;"AuthNegotiateDelegateWhitelist": "hador*.ics.muni.cz"<br/>}</tt> ** Windows: SPNEGO a Kerberos zřejmě není podporováno === curl === Příklady použití curl a SPNEGO. Detaily o souboru: curl -i --negotiate -u : "https://hador-c1.ics.muni.cz:50470/webhdfs/v1/<PATH>?op=LISTSTATUS" Stažení souboru: curl -L --negotiate -u : "https://hador-c1.ics.muni.cz:50470/webhdfs/v1/<PATH>?op=OPEN" -o file === Servisní stránky === '''Cluster''': * HDFS: https://hador-c1.ics.muni.cz:50470, https://hador-c2.ics.muni.cz:50470 * YARN: https://hador-c1.ics.muni.cz:8090, https://hador-c2.ics.muni.cz:8090 * MapRed History: https://hador-c2.ics.muni.cz:19890 * HBase: https://hador-c1.ics.muni.cz:60010 * Spark: http://hador-c2.ics.muni.cz:18080 '''Jednotlivé uzly''' (X=1..24): * https://hadorX.ics.muni.cz:50475/ - HDFS datanode * https://hadorX.ics.muni.cz:8044/ - YARN nodemanager * https://hadorX.ics.muni.cz:60030/ - HBase regionserver == NFS == Na '''hador.ics.muni.cz''' je HDFS lokálně připojeno přes NFS bránu do adresáře '''/hdfs'''. Pro HDFS-NFS bránu platí následující omezení: * je řádově pomalejší (cca 10⨯) * soubory nelze přepisovat = HBase = Viz [[Hadoop HBase]]. = Hive = == Informace == * správa přístupů: ** řešena na úrovni HDFS ==&gt; po vytvoření tabulky je možno řídit práva na HDFS adresáři /user/hive/warehouse/TABULKA ** výchozí práva na tabulce po vytvoření jsou včetně čtení a zápisu pro všechny uživatele Hadoopu == Připojení == * '''beeline''' klient ("$DBNAME" nahradit za název databáze): beeline -u "jdbc:hive2://hador-c1.ics.muni.cz:10000/$DBNAME;principal=hive/hador-c1.ics.muni.cz@ICS.MUNI.CZ" * beeline klient - připojení přes příkaz '!connect' ("DBNAME" nahradit za název databáze): beeline !connect jdbc:hive2://hador-c1.ics.muni.cz:10000/DBNAME;principal=hive/hador-c1.ics.muni.cz@ICS.MUNI.CZ '' '' * '''java''' kód: String url = "jdbc:hive2://hador-c1.ics.muni.cz:10000/DBNAME;principal=hive/hador-c1.ics.muni.cz@ICS.MUNI.CZ" Connection con = DriverManager.getConnection(url); * '''hive''' klient (deprecated, podporovaný je beeline klient): hive Detaily viz <tt>hive cli --help</tt>. == Příklady == * https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-DDLOperations DBNAME="`id -un`_test" JDBC_URL="jdbc:hive2://hador-c1.ics.muni.cz:10000/$DBNAME;principal=hive/hador-c1.ics.muni.cz@ICS.MUNI.CZ" # vyrobení databáze <USER>_test beeline -u $JDBC_URL -e "CREATE DATABASE $DBNAME" # použití <USER>_test beeline -u $JDBC_URL CREATE TABLE pokes (foo INT, bar STRING); CREATE TABLE invites (foo INT, bar STRING) PARTITIONED BY (ds STRING); SHOW TABLES; SELECT a.foo FROM invites a WHERE a.ds='2008-08-15'; DESCRIBE invites; INSERT INTO pokes VALUES (1, 'A'), (2, 'B'); INSERT INTO invites PARTITION (ds='2015-01-01') SELECT * FROM pokes; INSERT INTO invites PARTITION (ds='2015-12-11') SELECT * FROM pokes; = Pig = Spuštění: pig pig -e PŘÍKAZ pig -f SKRIPT == Příklady == fs -put /etc/passwd . A = load 'passwd' using PigStorage(':'); B = foreach A generate $0 as id; dump B; fs -rm ./passwd = Spark = == Informace == Podporovány režimy: * Spark přes YARN (cluster): --master yarn --deploy-mode cluster * Spark přes YARN (client): --master yarn --deploy-mode client * lokální Spark: --master local == Příklady == * výpočet pí v cluster režimu spark-submit --class org.apache.spark.examples.SparkPi --deploy-mode cluster --master yarn `ls /usr/lib/spark/lib/spark-examples-*.jar` 10 # (nahradit APPLICATION_ID za skutečné ID) yarn logs --applicationId APPLICATION_ID * shell v klient režimu hdfs dfs -put /etc/passwd # (nahradit LOGIN za uživatelské jméno) spark-shell --master yarn val file = sc.textFile("hdfs://hador-c1.ics.muni.cz:8020/user/LOGIN/passwd") val counts = file.flatMap(line => line.split(" ")).map(word => (word, 1)).reduceByKey(_ + _) counts.saveAsTextFile("hdfs://hador-c1.ics.muni.cz:8020/user/LOGIN/spark-output") hdfs dfs -cat ./spark-output/\* hdfs dfs -rm -r ./spark-output ./passwd efg26h271cawdjf3jwoadd5izbqx88l 0 458 5293 5273 2016-05-16T18:26:31Z 127.0.0.1 /* Advanced information */ wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[:Category:Applications|Available applications]]|}} {{BoxPoložka|[[Working with data]]|}} {{BoxPoložka|[[:Category:Scheduling system|Scheduling system]]|}} }} This guidepost provides the most important topics of ''grid computing'' offered by MetaCentrum. You might be interested in these topics: *[[How to compute/Quick start|Quick start]] - Quick tutorial with a simple example of computing in MetaCentrum ===Basic information=== First, you should read these topics to understand computing in MetaCentrum: *[[How to compute/Accessing machines|How to access machines]] - Available login methods *[[How to compute/Requesting resources|How to request resources]] - How to request offered by MetaCentrum *[[How to compute/Applications and modules|Applications and modules]] - How to use offered software (applications) in your computations ===Jobs=== Computing in MetaCentrum is possible via ''jobs'' (in combination with [[:Category:Scheduling system|''a scheduling system'']]), read more about types of jobs: *[[How to compute/Interactive jobs|Interactive jobs]] - Jobs with interactive approach without batch scripts *[[How to compute/Batch jobs|Batch jobs]] - Jobs based on batch scripts *[[How to compute/Job management|Job management]] - Topic describing tracking jobs, getting results and terminating jobs *[[How to compute/Troubleshooting|Troubleshooting]] - Topic providing information what to do if you encounter any problem ===Advanced information=== *[[How to compute/Parallelization|Parallelization]] - How to increase speed of your computations [[Category:Grid services]] ke13n5l579avo1iizrqmkhplluvg54w 0 459 5266 5232 2016-05-14T14:19:29Z Root 1 wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[Working with data/Data transfer|Data transfer]]|}} }} This page provides information about accessing MetaCentrum machines. First, select your '''operating system'''. {{YellowInfo|You have to be [[Application|registred]] to access the machines}} *[[How to compute/Accessing machines/From Windows|Accessing from Windows]] *[[How to compute/Accessing machines/From Linux|Accessing from Linux]] [[Category:Grid services]] osiprx2le01k5n39hhoczvv964zd57x 0 466 5268 1852 2016-05-14T14:25:31Z Root 1 Nahrazení textu „How to compute/Scheduled jobs“ textem „How to compute/Batch jobs“ wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[How to compute/Requesting resources|Requesting resources]]|}} {{BoxPoložka|[[Remote desktop]]|}} }} An interactive job is a special type of [[How to compute/Batch jobs|scheduled job]], but are working with granted resources rather than using a batch script. An interactive job is requested via '''qsub''' command and the '''-I''' option. For example: [[Image:InteraktivniUloha Pozadavek.png]] {{YellowInfo|Read more about qsub options [[How to compute/Requesting resources#Basic options|here]].}} Where '''-l''' are separators, '''walltime=1h''' means the job will last up to one hour, '''nodes=1:ppn=2''' is request on one machine with two processors, '''mem=64gb''' is request on 64 [https://en.wikipedia.org/wiki/Gibibyte GiB] of RAM memory. [[Category:Grid services]] *''Advantages of interactive jobs'' **GUI support **Better overview what I'm doing **Probably easier to learn *''Disadvantages of interactive jobs'' **Difficult to handle long-standing jobs (closing terminal will terminate the job) **Difficult to handle jobs working with multiple files **Internet connection reliability ==Graphical user interface== You can use graphical user interface ('''GUI''') while working with interactive jobs. These are available GUI options: * Recommended - [[Remote desktop|GUI environment via VNC servers]] * [[X-Window|Graphical tunnels via SSH]] q9kribkbdj1v5duu7ye2zowhrqi2tku 0 469 5239 1888 2016-05-13T14:27:49Z Root 1 wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[Application modules]]|}} {{BoxPoložka|[[:Category:Applications|Available applications]]|}} }} You can load an ''application'' offered by MetaCentrum to your job or machine via command '''module add''' + name of the selected application. If you are not sure what version of the application you would like to use, check the '''application's page''' first (you can use the search bar). For example: <source lang="bash"> module add matlab # loads Matlab application module add mrbayes-3.2.6 # loads MrBayes apllication, version 3.2.6 module list # shows currently loaded applications in your environment </source> Additional information about using the application, license etc. is available on the application's page. ==Installing a new application== If you would like to install a '''new application or new version of an application''', try to read [[How to install an application]] or contact [[:Category:User support|User support]]. [[Category:Grid services]] 8sper1ext3tkwggpxavnyu8xbtz7f9z 0 472 5274 5269 2016-05-14T14:27:27Z Root 1 wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[How to compute/Job management#Examine job's standard output and standard error output|Examining job's output]]|}} {{BoxPoložka|[[About scheduling system|Scheduling system]]|}} }} Batch jobs are a non-[[How to compute/Interactive jobs|interactive]] type of jobs. You must prepare a startup script (=''batch script'') for your job, submit the job, wait until it is finished and after that you can check its results. ==Prepare a startup script== {{Box|Related topics {{BoxPoložka|[[How to compute/Requesting resources|Requesting resources]]|}} {{BoxPoložka|[[Working with data/Working with data in a job|Working with data in a job]]|}} }} A startup script is collection of commands with given order, which are executed by your [[How to compute/Requesting resources|requested resources]]. You can specify requested resources '''outside''' (via qsub options) or '''inside''' your startup script. Both ways are absolutely correct, choose what you personally prefer. ===Specify requested resources outside the startup script=== Create the text file in your favourite text editor with following lines: <source lang="bash"> #!/bin/bash module add g03 #loads the Gaussian's application modules, version 03 cd $SCRATCHDIR #enters user's scratch directory wget http://www.cerit-sc.cz/en/docs/quickstart/gaussian_test.com #downloads job's input data g03 <gaussian_test.com >results.out #starts the Gaussian application and saves output into results.out cp results.out /home/$USER #moves the produced (output) data to user's home directory rm -rf $SCRATCHDIR #cleans the working directory </source> ===Specify requested resources inside the startup script=== You can specify resources inside the startup script via '''#PBS''' line prefix + an option. Create a text file in any editor with following lines: <source lang="bash"> #!/bin/bash #PBS -N myFirstJob #PBS -l walltime=1:00:00 #PBS -l nodes=1:ppn=4 #PBS -l mem=4gb #PBS -l scratch=10gb #Options above for scheduling system: job will run 1 hour at maximum, 1 machine with 4 processors + 4gb RAM memory + 10gb scratch memory are requested module add g03 #loads the Gaussian's application modules, version 03 cd $SCRATCHDIR #enters user's scratch directory wget http://www.cerit-sc.cz/en/docs/quickstart/gaussian_test.com #downloads job's input data g03 <gaussian_test.com >results.out #starts the Gaussian application and saves output into results.out cp results.out /home/$USER #moves the produced (output) data to user's home directory rm -rf $SCRATCHDIR #cleans the working directory </source> ==Submit the job== You can submit your job via '''qsub''' command, its options and your startup script name. Example: [[Image:qsubBatchRequest.png]] Qsub returns '''jobID''' (10583298 in this example) which you can use to [[How to compute/Job management#Tracking job status|track your job]]. If you specify requested resources inside your startup script, run just qsub command with name of the script. qsub myJob.sh ==Examine job's output== You can find information about this topic [[How to compute/Job management#Examine job's standard output and standard error output|here]]. [[Category:Grid services]] ldrm169sva7xjcd9w6c1ghfou676e52 0 475 2085 1872 2016-05-11T18:18:50Z Root 1 wikitext text/x-wiki PuTTY is an SSH and telnet client, developed originally by Simon Tatham for the Windows platform. ==Getting PuTTY== Visit PuTTy's [http://www.putty.org/ homepage] or go directly to the [http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html download page]. PuTTy is an installation-free software, just open the downloaded file. ==Configuring PuTTY== Step '''1''': Open the Session tab Step '''2''': Fill in target frontend address (list of frontends is avalaible on the [[Frontend|this]] page) [[Image:Putty1.png]] Step '''3''': Open the Connection/Data tab Step '''4''': Fill in your MetaCentrum username. [[Image:Putty2.png]] Step '''5''': Open the Connection/SSH tab Step '''6''': Select 2 only as the preferred SSH protocol version [[Image:Putty3.png]] Step '''7''': Open the Connection/SSH/Auth/GSSAPI Step '''8''': Check in both options [[Image:Putty4.png]] Step '''9''': Open the Session tab Step '''10''': Fill in the name of the current configuration Step '''11''': Click Save [[Image:Putty5.png]] Finally, click '''Open''', a terminal should appear. [[Category:Grid services]] kewkeqfd0drqz2sdv2hcb5o0cj39sjs 0 484 1870 1869 2016-04-28T08:49:13Z Root 1 /* Usage */ wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[Working with data/Data transfer#Data transfer on Windows|Data transfer on Windows]]|}} }} You can choose one the methods below to access machines from Windows. '''The SSH client method is recommended for new users'''. ==Using SSH client== You need an application which emulates terminal with ssh client to access MetaCentrum machines. We recommend one of these applications: *[[Usage of PuTTY|PuTTY]] *[[Usage of Bitvise SSH Client|Bitvise SSH Client]] *[[Usage of WinSCP|WinSCP]] ==Using Kerberos== {{RedInfo|This method is recommended for advanced users}} {{Box|Related topics {{BoxPoložka|[[Kerberos authentication system]]|}} }} These tutorials work on Windows 8.1, other versions of Windows may need slight changes. ===Installation=== '''1.''' Download & install Kerberos. Download the latest Kerberos release from the [http://web.mit.edu/Kerberos/dist/index.html#kfw-4.0 MIT page] and install it. '''2.''' Enable hidden items You have to enable hidden items in order to see the ProgramData folder needed in the next step. [[Image:WindowsKerberos1.png]] '''3.''' Edit kr5.ini file Download/copy one of these files depending on your location: * If you are closest to ''Brno'' - http://meta.cesnet.cz/metafiles/krb5.conf.muni * If you are closest to ''Prague'' - http://meta.cesnet.cz/metafiles/krb5.conf.cuni * If you are closest to ''Pilsen'' - http://meta.cesnet.cz/metafiles/krb5.conf.zcu {{YellowInfo|Save the file as '''"'''krb5.ini'''"''' with quotation marks to force .ini file extension}} and save it as/to C:\ProgramData\MIT\Kerberos5\'''krb5.ini''' (you need administrative privileges to do so). ===Usage=== '''1.''' Open MIT Kerberos [[Image:WindowsKerberos2.png]] '''2.''' Get a new ticket [[Image:WindowsKerberos3.png]] '''3.''' Fill in MetaCentrum username and password [[Image:WindowsKerberos4.png]] and click '''OK''' You should see a valid Kerberos ticket now and if you did everything right, you should not need to type in your password during login phase (e.g. when using [[Usage of PuTTY|PuTTY]]): [[Image:WindowsKerberos5.png]] [[Category:Grid services]] ix5hqxdf4hiezkqpgpua5jy81mohbjs 0 496 1805 1779 2016-04-04T15:21:28Z Root 1 wikitext text/x-wiki {{Box|Related topics {{BoxPoložka|[[:Category:User support|User support]]|}} {{BoxPoložka|[[Reasons for an unusual job ending]]|}} }} Try to follow these steps if you encounter any problem: '''1.''' Check MetaCentrum documentation * Have a look at [[FAQ]] * If you have the problem with an applications try checking [[:Category:Applications|application]] module documentation first '''2.''' Check if there are not any undergoing updates/outages * See http://metavo.metacentrum.cz/en/news/outages.jsp for ''outages'' * See http://metavo.metacentrum.cz/en/news/news.jsp for ''updates'' '''3.''' Contact user support * Write an email to meta@cesnet.cz - your email will create a ticket in our Request Tracking system Please provide '''as much information as possible''' - problematic jobID, your startup script, target machine etc. [[Category:Grid services]] cp3s7r4mmtqn51aymbhd60dturq2gkp 0 471 5307 5306 2016-05-17T21:08:51Z Root 1 wikitext text/x-wiki __NOTOC__ {{Box|Related topics {{BoxPoložka|[[How to compute]]|}} }} This Step by Step tutorial shows how to run ''a job'' on MetaCentrum machines, see the [[How to compute]] topic for more information or follow steps bellow to run an example job. You can also see [https://www.youtube.com/watch?v=uR3FDgo01lc '''video tutorial'''], which covers this topic (works with PuTTY and Windows). {{YellowInfo|A terminal is required, Windows users might want to use [[Usage of PuTTY|PuTTY]].}} ==1. Log on a frontend== {{Box|Related topics {{BoxPoložka|[[How to compute/Accessing machines|Accessing machines]]|}} }} Type the line below into your terminal and fill in your META password: <source lang="bash"> ssh 'META username'@skirit.metacentrum.cz Password: <Enter META password> </source> ==2. Prepare a startup script== {{Box|Related topics {{BoxPoložka|[[How to compute/Batch jobs|Batch jobs]]|}} }} Create ''myJob.sh'' text file (in your favourite '''text editor''') with the following lines: <source lang="bash"> #!/bin/bash # sets home directory DATADIR="/storage/brno2/home/$LOGNAME/" # loads the Gaussian's module, version 03 module add g03 # enters user's scratch directory cd $SCRATCHDIR # prepares job's input data wget http://www.cerit-sc.cz/en/docs/quickstart/gaussian_test.com # starts the Gaussian application g03 <gaussian_test.com >results.out # moves the produced (output) data to user's home directory cp results.out $DATADIR # cleans the working directory rm -rf $SCRATCHDIR </source> ==3. Specify job's parameters & submit the job== {{Box|Related topics {{BoxPoložka|[[How to compute/Requesting resources|Requesting resources]]|}} }} Type the following line into your terminal: <source lang="bash"> qsub -l walltime=1:00:00 -l nodes=1:ppn=2 -l mem=1gb -l scratch=10gb myJob.sh </source> You should see an output with '''jobID''' such as 12345.arien.ics.muni.cz. JobID is ''12345'' in this case. ==4. Track job's status== {{Box|Related topics {{BoxPoložka|[[How to compute/Job management#Tracking job status|Tracking job status]]|}} }} You can use ''jobID'' and ''qstat'' command to track the job. Type line below into your terminal: <source lang="bash"> qstat 'jobID'.arien.ics.muni.cz # jobID in our example is 12345 </source> You should see an output similar to these lines: <source lang="bash"> Job id Name User Time Use S Queue ------------------------- ---------------- --------------- -------- - ----- 12345.arien myjob.sh 'your username' 00:14:06 C q_2h # C=Completed, W=Waiting, Q=queued </source> ==5. Examine job's standard output and standard error output== {{Box|Related topics {{BoxPoložka|[[How to compute/Job management#Examine job's standard output and standard error output|Examining outputs]]|}} }} When the job is completed you can examine it's outputs in your terminal: <source lang="bash"> cat ~/results.out # job's output data cat ~/myjob.sh.o12345 # job's standard output, empty for now (forwarded to results.out) cat ~/myjob.sh.e12345 # job's standard error output </source> [[Category:Grid services]] 3g0l4pwd82gpdsbhpd1ymzo3379blz2 10 62 224 223 2016-01-10T12:56:53Z Root 1 naimportována 1 revize wikitext text/x-wiki <div style="border:1px solid red;background-color:#f9e9e9;padding:3px;margin:5px;"> [[Image:Warning.gif]] '''WARNING:''' {{{1}}} </div> nethgdh95ae3lhw1zfk4uvk647v2kbo 10 405 921 920 2016-01-23T18:50:50Z Root 1 naimportována 1 revize wikitext text/x-wiki {| cellspacing="3" |- valign="top" |valign="top" colspan="2" width="{{{width|100%}}}" class="MainPageBG" style="border: 1px solid {{{solid|#00ff00}}}; color: {{{color|#000000}}}; background-color: {{{background-color|#b0ffb0}}}"| <div style="padding: .4em .9em .4em" align="{{{align|center}}}">{{{text|Chybí zadána povinná položka 'text'.}}}</div> |} dazxa51z0v6q8let5jc4820aeqn1ila 10 6 2094 2093 2016-05-12T09:33:43Z Root 1 wikitext text/x-wiki {| cellpadding="0" align="right" class="tabulka" |- style="text-align:center;color:white;" ! colspan="2" style="background: {{{barva|DarkGreen}}};" | '''{{{1|{{PAGENAME}}}}}''' |} 8bhwddxzh7wq73tbwtwzewlgm1lkres 10 15 43 2015-10-04T13:23:57Z Root 1 Založena nová stránka s textem „{{BoxŘádek|typ={{{polozka|{{{1}}}}}}|hodnota={{{hodnota|{{{2}}}}}}|detail=|}}“ wikitext text/x-wiki {{BoxŘádek|typ={{{polozka|{{{1}}}}}}|hodnota={{{hodnota|{{{2}}}}}}|detail=|}} o1tsryojr4q2e7ya4bjal2xh9ujlnh2 10 14 945 137 2016-01-25T09:07:32Z Root 1 wikitext text/x-wiki |- valign=top ! align=left style="border: 0px; span style="color:white background: {{{barva|White}}};" | {{{typ}}} | align=left style="border: 0px;" | {{{hodnota}}} {{{detail}}} ja1kidtr39wztchchn42ppnlwy6yq9y 10 570 1851 1850 2016-04-26T14:19:55Z Root 1 wikitext text/x-wiki <div style="border:1px solid green;background-color:#b0ffb0;padding:3px;position:relative; margin:auto; width:750px"> {{{1}}} </div> ie0h80ij814w4rcu70uxl2eeqw22gli 10 406 923 922 2016-01-23T18:50:50Z Root 1 naimportována 1 revize wikitext text/x-wiki {{Rámeček|text={{{text}}}|solid=#00ff00|color=#000000|background-color=#f3fff3|align=left}} 8zmtyx497ljoqn8t2dmxq45kd0na4of 10 407 925 924 2016-01-23T18:50:50Z Root 1 naimportována 1 revize wikitext text/x-wiki '''Notice:''' To use the application you first need to '''[http://metavo.metacentrum.cz/en/myaccount/licence.html accept it's licence]'''. azyyy6wemmgmfq5ap9asgsmj9ywzf96 10 430 1017 1016 2016-01-26T13:01:20Z Root 1 naimportována 1 revize wikitext text/x-wiki <code class="plainlinks">[{{{JavaDocLink|http://docs.oracle.com/javase/7/docs/api/}}}{{{package|java/lang}}}/{{{class}}}.html#{{{method}}} {{{classPrefix|}}}{{{class}}}.{{{method}}}]</code> maochpd9bey97qbxb7uh3k2mcvi20h1 10 461 1213 1181 2016-02-13T17:49:19Z Root 1 wikitext text/x-wiki <div style="border:1px solid yellow;background-color:LightYellow;padding:3px;position:relative; margin:auto; width:750px"> [[Image:zarovkaMala.png]] '''Note:''' {{{1}}} </div> ra8e5el5ba1qkhi9gmxtwagaz3qk38q 10 483 1340 2016-02-23T11:32:20Z Root 1 Založena nová stránka s textem „<div style="border:1px solid red;background-color:#f9e9e9;padding:3px;position:relative; margin:auto; width:750px"> [[Image:Warning.gif]] '''Warning:''' {{…“ wikitext text/x-wiki <div style="border:1px solid red;background-color:#f9e9e9;padding:3px;position:relative; margin:auto; width:750px"> [[Image:Warning.gif]] '''Warning:''' {{{1}}} </div> d5x088xiu21heejuqyzv571yxxundsm 0 528 1843 1453 2016-04-19T14:14:22Z Root 1 wikitext text/x-wiki {| class="wikitable" ! style="background-color:LightSeaGreen;"|NFS4 server !! style="background-color:LightSeaGreen;"|Directory !! style="background-color:LightSeaGreen;"|Size !! style="background-color:LightSeaGreen;"|Alternative server names |- | '''storage-brno2.metacentrum.cz''' || /storage/brno2/ || 110 TB || <nowiki>(nienna1|nienna2|nienna-home)</nowiki>.ics.muni.cz |- | '''storage-brno3-cerit.metacentrum.cz''' || /storage/brno3-cerit/ || 260 TB || kefeus-home.cerit-sc.cz |- | '''storage-brno4-cerit-hsm.metacentrum.cz''' || /storage/brno4-cerit-hsm/ ||512 TiB || nfs1-hsm.cerit-sc.cz |- | '''storage-brno5-archive.metacentrum.cz''' || /storage/brno5-archive/ || 4PB || nfs.du3.cesnet.cz |- | '''storage-brno6.metacentrum.cz''' || /storage/brno6/ || 262 TB || |- | '''storage-brno7-cerit.metacentrum.cz''' || /storage/brno7/ || 374 TB || fineus-home.cerit-sc.cz |- | '''storage-brno8.metacentrum.cz''' || /storage/brno8/ || 88 TB || original /storage/ostrava1/ |- | '''storage-brno9-ceitec.metacentrum.cz''' || /storage/brno9-ceitec/ || 262 TB || storage-ceitec1.ncbr.muni.cz |- | '''storage-budejovice1.metacentrum.cz''' || /storage/budejovice1/ || 44 TB || <nowiki>(storage-cb1|storage-cb2)</nowiki>.metacentrum.cz |- | '''storage-jihlava2-archive.metacentrum.cz''' || /storage/jihlava2-archive/ ||4266 TiB || nfs.du2.cesnet.cz |- | '''storage-plzen1.metacentrum.cz''' || /storage/plzen1/ || 350 TB || <nowiki>(storage-eiger1|storage-eiger2|storage-eiger3)</nowiki>.zcu.cz |- | '''storage-plzen2-archive.metacentrum.cz''' || /storage/plzen2-archive/ ||8034 TiB || nfs.du1.cesnet.cz |- | '''storage-praha1.metacentrum.cz''' || /storage/praha1/ || 51 TB || <nowiki>storage-praha1(a|b)</nowiki>.metacentrum.cz |} neb6sxfbh75ilvz34cicg7se6z2d793 10 431 1019 1018 2016-01-26T13:01:20Z Root 1 naimportována 1 revize wikitext text/x-wiki {| class="tabulka" width="25%" align="right" ! [[Image:Info_red.png|25px]] ! Tomuto dokumentu něco chybí! |- | colspan="2" | {{{1|Není zadán přesný popis problému v dokumentu. Prosím, podívejte se, zda dokument odpovídá aktuálnímu stavu či stylu a případně jej opravte.''}}} |- | colspan="2" | ''Pokud jste dokument opravili, odstraňte řádek se šablonou [[Šablona:TODO|TODO]] z tohoto dokumentu.'' |} <noinclude> Tento template se používá na dokumenty, které neodpovídají stylu nebo jsou problémové z jiného hlediska a je třeba, aby byly opraveny. Jediným parametrem je text, kterým se informuje o problému v dokumentu. </noinclude> olzmeuhw68vc4ri16npft6qzs56uqfg 10 130 376 375 2016-01-23T15:51:08Z Root 1 naimportována 1 revize wikitext text/x-wiki <div style="border:solid thin gray;background-color: whitesmoke;padding:3px;margin:5px;"> [[Image:information.png]] '''Please note:''' {{{1}}} </div> njffmujrrz9l8r06es32ovui7uu1ba9 10 131 378 377 2016-01-23T15:51:08Z Root 1 naimportována 1 revize wikitext text/x-wiki [{{{PerunLink|https://perun.metacentrum.cz/perun-gui/#}}}{{{rel}}}] kc36jbxe3ebz7fz92bbatho0uagc1iq 10 132 380 379 2016-01-23T15:51:09Z Root 1 naimportována 1 revize wikitext text/x-wiki [[Perun_system|Back to Perun main page]] 2jydxan3dadu81pchtehb26j773fbdy 10 133 382 381 2016-01-23T15:51:09Z Root 1 naimportována 1 revize wikitext text/x-wiki <div style="border:solid thin gray;background-color: whitesmoke;padding:3px;margin:5px;"> [[Image:disk32.png]] '''Please note:''' {{{1}}} </div> 5snvwl6810mor2wyldgzt58fciqzmz2