The Wayback Machine - https://web.archive.org/web/20220112051501/https://github.com/kedro-org/kedro-viz/pull/681
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve CI Setup #681

Merged
merged 22 commits into from Jan 5, 2022
Merged

Improve CI Setup #681

merged 22 commits into from Jan 5, 2022

Conversation

@limdauto
Copy link
Contributor

@limdauto limdauto commented Jan 5, 2022

Description

  • Use the newer cimg image as opposed to the legacy circleci.
  • Run npm install instead of npm ci to allow node_modules cache to take effect
  • Remove conda in linux build. Just standard python venv is enough.
  • Cache pip & python virtualenvs between builds.
  • Parallelise make pylint

Net result

  • Linux build: 11m25s -> 7m35s (35%)
  • Windows build: 11m47s -> 9m37s (18%)

Before:

Screenshot 2022-01-05 at 16 32 19

After:

Screenshot 2022-01-05 at 16 31 47

Further improvement

Maybe we can run windows test on main only.

Development notes

QA notes

Checklist

  • Read the contributing guidelines
  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added new entries to the RELEASE.md file
  • Added tests to cover my changes
Copy link
Member

@tynandebold tynandebold left a comment

Looks good to me. The results speak for themselves. Nice work.

Just one comment about the tools/ci.js file, but I'll approve straight away.

@@ -51,7 +43,7 @@ commands:
steps:
- run:
name: Install Node dependencies
command: node tools/ci.js
command: npm install
Copy link
Member

@tynandebold tynandebold Jan 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can delete the tools/ci.js file now. Doesn't seem to be used anywhere else.

@limdauto limdauto merged commit 72b47ff into main Jan 5, 2022
11 checks passed
@limdauto limdauto deleted the fix/ci-python-cache branch Jan 5, 2022
Copy link
Contributor

@studioswong studioswong left a comment

Nice one @limdauto - thanks for setting this up!

# Get rid of pyenv stuff
sudo rm -rf .pyenv/ /opt/circleci/.pyenv/
# Download and install miniconda
curl https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh > miniconda.sh
bash miniconda.sh -b -p $HOME/miniconda
# Create an anaconda virtualenv for python ${CONDA_ENV_PY_VERSION} and make that the default python interpreter
echo ". /home/circleci/miniconda/etc/profile.d/conda.sh" >> $BASH_ENV
echo "conda deactivate; conda activate kedro-viz" >> $BASH_ENV
. /home/circleci/miniconda/etc/profile.d/conda.sh
conda create --name kedro-viz python=${CONDA_ENV_PY_VERSION} -y
source $BASH_ENV
Copy link
Contributor

@studioswong studioswong Jan 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@limdauto this is more for my curiosity - I wonder why won't we need conda as the default python interpreter with the new setup? Is it becuase of the switch to use cimg?

rashidakanchwala added a commit that referenced this issue Jan 10, 2022
Signed-off-by: Rashida Kanchwala <rashida.kanchwala@quantumblack.com>
rashidakanchwala added a commit that referenced this issue Jan 11, 2022
Signed-off-by: Rashida Kanchwala <rashida.kanchwala@quantumblack.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
3 participants