Artifactory Tech Talk Notes
Introduction to Artifactory
Where is it?
What is it?
Essentially GitHub for binary files.
A binary repository manager is a software tool designed to optimize the download and storage of binary files used and produced in software development. It centralizes the management of all the binary artifacts generated and used by the organization to overcome the complexity arising from the diversity of binary artifact types, their position in the overall workflow and the dependencies between them.
A binary repository is a software repository for packages, artifacts and their corresponding metadata. It can be used to store binary files produced by an organization itself, such as product releases and nightly product builds, or for third party binaries which must be treated differently for both technical and legal reasons.
How does it work?
Artifactory Pro provides a number of predefined repository templates that all follow the same pattern with a set of three repositories:
A remote repository which provides indirect access to the public repository. For example, there’s a
docker-remoterepository that provides access to Docker Hub through Artifactory. When you download an image, such as the community golang image, for the first time Artifactory allows you to access it via the remote repository and stores it in docker-remote-cache for future requests.
A local repository which stores artifacts that you upload to artifactory either directly or as the output of a build. In the case of Docker, if you build a new image based on the golang image and push it to Artifactory, it’s saved in the
A virtual repository which combines the remote and local repository so that clients can request artifacts from the repository without needing to know if they are available locally or remotely.
Pulling an existing image from the Artifactory Docker repository
- Verify that you can contact the Artifactory server
curl -I -k -v http://b4v7eywig.activestate.com:8081/artifactory/api/system/ping
- Log in
docker login -u admin -p thisisatest123 b4v7eywig.activestate.com:8081
- Pull an image
docker pull b4v7eywig.activestate.com:8081/docker/active-python-3-5-4:latest
Pulling an image that isn’t in the local Artifactory Docker repository
docker pull b4v7eywig.activestate.com:8081/docker/golang
In this case the first time the image is pulled it comes from
library\golang on Docker Hub and is stored in
library\golang in Artifactory’s
docker-remote-cache repository. Future pulls of the same image will come from the cache.
Tagging an updated image and pushing to the repository
- Make your changes and build the new image
docker build --rm -t b4v7eywig.activestate.com:8081/docker/active-python-3-5-4:3.0 -t b4v7eywig.activestate.com:8081/docker/active-python-3-5-4:latest .
- Push the updates to Artifactory
docker push b4v7eywig.activestate.com:8081/docker/active-python-3-5-4:3.0
docker push b4v7eywig.activestate.com:8081/docker/active-python-3-5-4:latest
Note: You need to configure the Artifactory registry as an insecure registry since the trial instance isn’t using ssl. When configured for production the commands would be more like typical docker hub commands, for example:
docker push hub.activestate.com/docker/active-python-3-5-4:latest
Python Package Index (PyPI)
Each Artifactory repository has a “Set me up” link that provides details about the configuration changes you need to make.
pip install pythonPT --trusted-host b4v7eywig.activestate.com pip install pythonPT==0.1.0 --trusted-host b4v7eywig.activestate.com pip install pythonPT==0.1.1 --trusted-host b4v7eywig.activestate.com pip install bleach --trusted-host b4v7eywig.activestate.com
pythonPT is a custom package so it is stored in the local repository. bleach comes from PyPi via the remote repository and is stored in pypy-remote-cache for future requests. To the end user it looks the same. They just need to do a standard pip install.
Note: Once it’s configured Pip works just like it would if it was pointing to the PyPi repository. The
--trusted-host flag is only required because we don’t have ssl configured on the trial instance.
Go and vgo
Vgo is the proposed official standard dependency management tool and registry specification for Go. While not yet included in the official Go tools set, it is poised to be the standard dependency manager distributed with each Go release. Artifactory supports vgo though its
jfrog cli tool. It fills in a few gaps that currently exist in the vgo CLI. JFrog CLI wraps vgo and adds additional capabilities with simple commands.
- Local repositories in Artifactory let you set up secure, private Go registries with fine-grained access control to packages according to projects or development teams.
- A remote repository in Artifactory is a caching proxy for remote Go resources such as a GitHub project. Accessing a go proxy through Artifactory removes your dependency on the network, or on GitHub since all dependencies needed for your Go builds are cached in Artifactory and are therefore locally available. This also removes the risk of someone mutating or removing a dependency from version control, or worse, force-pushing changes to a remote Git tag thus changing what should be an immutable version which can create a lot of confusion and instability for dependant projects.
- A virtual repository aggregates local and remote registries
- Install go and vgo
- Install the jfrog cli:
curl -fL https://getcli.jfrog.io | sh
- Configure the connection information for the Artifactory server
- Use the
jfogcli to build and deploy your release
jfrog rt config ## Use jfrog rt config show to display an existing configuration jfrog rt go build --no-registry jfrog rt gp go-local --self=false --deps=ALL jfrog rt go build go-virtual jfrog rt gp go-local v1.0.0 --build-name=my-build --build-number=1 jfrog rt bce my-build 1 jfrog rt bp my-build 1
Perl and CPAN:
Artifactory does not have support for CPAN. It’s been requested but the best they can do right now is cache modules that have been requested from CPAN. Possibly the same can be done for Tcl.
[CircleCI project]() Github repo
You can adding jFrog cli commands to the CircleCI configuration file (
.circleci/config.yaml) publish builds to Artifactory.
- Download the jFrog cli
- Configure the Artifactory connection. Uses environment variables configured in
- Run NPM install (npmi).
- Collect environment variables.
- Pack and deploy the npm package to the designated npm repository.
- Publish build info
- run: curl -fL https://getcli.jfrog.io | sh - run: ./jfrog rt config --url $ARTIFACTORY_URL --user $ARTIFACTORY_USER --password $ARTIFACTORY_PASSWORD --interactive=false - run: rm -rf node_modules/ - run: ./jfrog rt npmi npm --build-name=$CIRCLE_JOB --build-number=$CIRCLE_BUILD_NUM - run: ./jfrog rt bce circleci-npm-artifactory $CIRCLE_BUILD_NUM - run: ./jfrog rt npmp npm --build-name=circleci-npm-artifactory --build-number=$CIRCLE_BUILD_NUM - run: ./jfrog rt bp circleci-npm-artifactory $CIRCLE_BUILD_NUM
- run: curl -fL https://getcli.jfrog.io | sh - run: ./jfrog rt config --url $ARTIFACTORY_URL --user $ARTIFACTORY_USER --password $ARTIFACTORY_PASSWORD --interactive=false - run: ./jfrog rt bce circleci-python-artifactory $CIRCLE_BUILD_NUM - run: ./jfrog rt bp circleci-python-artifactory $CIRCLE_BUILD_NUM
SWOT (or how does it affect ActiveState?)
Strengths (that we can take advantage of)
- Focuses on a specific problem and appears to do it well
- Standardizes the way you interact with a wide range of package managers and open source repositories. Provides one place to manage a wide variety of artifacts.
- Supports a variety of use cases from open source to high-availbility, global deployments
- No specific support of Perl (CPAN) or Tcl (Gutter) repositories. For Perl it is possible to configure a generic repository to cache modules after they are downloaded from CPAN.
- Doesn’t tell administrators why is a package needed (i.e. what apps use a package, who will care if it’s removed)
- Accelerate Platform development by providing some of the low level plumbing for interacting with repositories, archiving builds, and managing artifacts.
- Provides a complete API and CLI client that we can integrate with
- jFrog doesn’t seem too focused on the individual developer beyond supporting the various package management tools. More devops focused.
Threats (from jFrog as a competitor)
- jFrog has established a foothold in lots of companies
- They’re building a platform that is capable of complementing the ActiveSate Platform, but is also in some feature areas a direct competitor. For example, they have a product for scanning binaries (X-Ray) and also have a Black Duck integration.
- What To Think About When Thinking About Onboarding Artifactory
- Running an In-house Go Registry with Artifactory
- Artifactory Docker Registries Feel Like Home Manage Your Docker Builds with JFROG CLI in 5 Easy Steps!
- Interesting to look at how they’ve segmented paid vs free/OSS features and Pro vs. Enterprise
- Support for on prem, hosted, and cloud deployments
- Integration with existing tooling (e.g. pip), jfrog cli, and API access
“If you do not have a CI integration or your organizational procedures require you to configure package managers and build tools on individual workstations, Artifactory provides you with an easy to use Set Me Up feature. This feature will auto-generate the configuration snippets and commands you need.
If you have a CI/CD setup, you can integrate Artifactory into your build ecosystem and gain visibility of artifacts deployed, dependencies and information on the build environment. You can connect your workstations to Artifactory, with all the necessary configuration of the various package managers and clients, using our CI server integrations.”
- Setup Artifactory alongside your current system.
- Configure your builds to start pushing content, including builds and artifacts, to Artifactory while still maintaining your old system. i.e retrieve/deploy to your legacy system but in parallel also deploy to Artifactory.
- Start deploying and retrieving content directly to/from Artifactory. But at the same time continue to deploy your content to the legacy system, for the consumption of other teams/projects.
- Once you are ready and confident with your Artifactory integration, start migrating your projects one by one or in bulk.
Some of the inherent added value of introducing Artifactory into your environment is improving the build process. For example, utilizing your build info and metadata will allow you to have traceable builds and easily configured retention procedures.
Having traceable builds will prevent the delay of build process by stages like compliance, security and release management. On the one hand, it will allow you to separate the development process from the post-development stages such as QA, compliance, and security. And on the other hand still maintain bidirectional connection between all of this post-development steps.