Skip to content

Make RELATE language-agnostic by allowing specification of Docker containers. - #674

Merged
inducer merged 12 commits into
inducer:masterfrom
UI-CS101:dockercodequestion
Nov 4, 2019
Merged

inducer merged 12 commits into
inducer:masterfrom
UI-CS101:dockercodequestion

Conversation

@davis68

@davis68 davis68 commented Sep 18, 2019

Copy link
Copy Markdown
Contributor

This contributes further towards #631, creating support for non-Python questions.

The current version (c6943a7) adds support for specifying Docker containers, although the default on everything is still Python. Terminology such as runpy has been switched to runcode. The Docker image inducer/relate-runcode-python has been introduced.

@codecov

codecov Bot commented Sep 18, 2019 •

Copy link
Copy Markdown

Codecov Report

Merging #674 into master will increase coverage by 7.81%.
The diff coverage is 60%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #674      +/-   ##
==========================================
+ Coverage   88.42%   96.23%   +7.81%     
==========================================
  Files          45       45              
  Lines       11117    11217     +100     
  Branches     2065     2085      +20     
==========================================
+ Hits         9830    10795     +965     
+ Misses       1097      327     -770     
+ Partials      190       95      -95
Impacted Files Coverage Δ
course/page/code.py 98.3% <60%> (+59.13%) ⬆️
relate/utils.py 85.64% <0%> (-8.62%) ⬇️
course/versioning.py 88.03% <0%> (-8.23%) ⬇️
course/constants.py 100% <0%> (ø) ⬆️
relate/urls.py 100% <0%> (ø) ⬆️
course/validation.py 99.81% <0%> (ø) ⬆️
course/content.py 96.69% <0%> (+0.63%) ⬆️
accounts/models.py 100% <0%> (+1.03%) ⬆️
course/models.py 97.09% <0%> (+1.05%) ⬆️
course/utils.py 98.83% <0%> (+1.66%) ⬆️
... and 16 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3f7ea30...aebd15d. Read the comment docs.

@davis68

davis68 commented Sep 18, 2019

Copy link
Copy Markdown
Contributor Author

Looks like it's mad about the CSV import error you've fixed on another branch, but that looks like all of it.

Ah, you'll have to patch back in this one or I can rebuild the PR once you've merged #667.

@inducer

inducer commented Sep 19, 2019

Copy link
Copy Markdown
Owner

#675 should have cleared up all the CI failures. (Sorry about having those hanging for so long.) Could you merge in master and see what happens?

Comment thread tests/test_grades/test_csv.py Resolved
@davis68

davis68 commented Sep 19, 2019 •

Copy link
Copy Markdown
Contributor Author

BUT it's going to be pulling in the default Docker image, not the new one. See docker_image_run_py/Dockerfile for details. We should definitely test that before a merge. It works locally but that's worth about the letters it takes to type it.

@inducer

inducer commented Sep 20, 2019

Copy link
Copy Markdown
Owner

I've built and pushed inducer/relate-runcode-python. Could you apply this diff:

diff --git a/docker-image-run-py/Dockerfile b/docker-image-run-py/Dockerfile
index 4e9eacf2..f73228e1 100644
--- a/docker-image-run-py/Dockerfile
+++ b/docker-image-run-py/Dockerfile
@@ -3,9 +3,6 @@ MAINTAINER Andreas Kloeckner <inform@tiker.net>
 EXPOSE 9941
 RUN useradd runcode
 
-# Temporarily needed for pandas
-RUN echo "deb http://httpredir.debian.org/debian unstable main contrib" >> /etc/apt/sources.list
-RUN echo "deb http://httpredir.debian.org/debian experimental main contrib" >> /etc/apt/sources.list
 RUN echo 'APT::Default-Release "testing";' >> /etc/apt/apt.conf
 
 RUN apt-get update
@@ -40,8 +37,6 @@ RUN apt-get -y -o APT::Install-Recommends=0 -o APT::Install-Suggests=0 install \
 RUN apt-get clean
 RUN fc-cache
 
-RUN pip3 install numpy==1.16.1
-
 RUN mkdir -p /opt/runcode
 ADD runcode /opt/runcode/
 COPY code_feedback.py /opt/runcode/
@@ -53,7 +48,8 @@ RUN echo "savefig.dpi : 80" >> /etc/matplotlibrc
 RUN echo "savefig.bbox : tight" >> /etc/matplotlibrc
 
 # RUN pip3 install --upgrade tensorflow
+# RUN  find /usr/lib/python3/ -name '*cpython-37m*.so' -delete
+
 RUN rm -Rf /root/.cache
-RUN  find /usr/lib/python3/ -name '*cpython-37m*.so' -delete
 
 # may use ./flatten-container.sh to reduce disk space

(and, for future PRs (and maybe this one), could you tick the box that allows me to push?)

@inducer

inducer commented Sep 20, 2019

Copy link
Copy Markdown
Owner

Should this also include changing the name of the setting RELATE_DOCKER_RUNPY_IMAGE and pointing at the new container?

@davis68

davis68 commented Sep 20, 2019 •

Copy link
Copy Markdown
Contributor Author

The box Allow edits from maintainers is checked. What do you observe that is different with this PR's behavior than what you expect?

@davis68

davis68 commented Sep 20, 2019

Copy link
Copy Markdown
Contributor Author

Should this also include changing the name of the setting RELATE_DOCKER_RUNPY_IMAGE and pointing at the new container?

Yes. My local_settings.py has:

RELATE_DOCKER_RUNPY_IMAGE = "inducer/relate-runcode-python"

(and will have RELATE_DOCKER_RUNOC_IMAGE = "davis68/relate-runcode-octave" later).

@davis68

davis68 commented Sep 20, 2019

Copy link
Copy Markdown
Contributor Author

I should probably hardcode a fallback in case of an older local_settings.py.

@inducer

inducer commented Sep 22, 2019

Copy link
Copy Markdown
Owner

RELATE_DOCKER_RUNOC_IMAGE

Could you spell out RUNOCTAVE? (I'm happy to have Python also spelled out for fairness. "OC" is a bit hard to guess otherwise...)

@inducer

inducer commented Sep 22, 2019

Copy link
Copy Markdown
Owner

Should this also include changing the name of the setting RELATE_DOCKER_RUNPY_IMAGE and pointing at the new container?

Yes.

That's my question. Should I wait with merging this until those changes are included? I.e. could you let me know when/whether this is ready for review?

@davis68

davis68 commented Oct 1, 2019

Copy link
Copy Markdown
Contributor Author

Current major references to RELATE_DOCKER_RUNPY_IMAGE include:

  • /tests/test_pages/test_code.py has an override (see REAL_ below)
  • course/page/code.py loads it, of course

Current hardcoded references to inducer/relate-runpy-* include:

  • tests/test_pages/utils.py refers to REAL_RELATE_DOCKER_RUNPY_IMAGE as inducer/relate-runpy-i386
  • local_settings_example.py has the old version

I have just updated the Dockerfile per your diff above and also fixed local_settings_example.py. If these clear, I am "reasonably confident" that things have integrated successfully, barring what the i386 version does (which I have not investigated).

@inducer My actionable question (assuming checks pass) is whether to swap out the hardcoded i386 version for the generic runcode version of the image.

@davis68

davis68 commented Oct 10, 2019

Copy link
Copy Markdown
Contributor Author

@inducer ☝️ Please see the question on the hardcoded i386 Docker image.

@inducer

inducer commented Oct 10, 2019

Copy link
Copy Markdown
Owner

My actionable question (assuming checks pass) is whether to swap out the hardcoded i386 version for the generic runcode version of the image.

Yes, please do. Sorry about the delay in responding. I've had a pretty rough two weeks, and I'm still battling a cold...

@davis68

davis68 commented Oct 11, 2019

Copy link
Copy Markdown
Contributor Author

No problem. I think I may be getting the cold going around myself...

@davis68

davis68 commented Oct 30, 2019

Copy link
Copy Markdown
Contributor Author

Bumping this, then I'll finalize the Octave addition and write up documentation.

Comment thread course/page/code.py Resolved
@inducer

inducer commented Oct 31, 2019

Copy link
Copy Markdown
Owner

Sorry for the delay! Just one minor doc wrinkle to go.

@inducer
inducer merged commit 63a2fb6 into inducer:master Nov 4, 2019
@inducer

inducer commented Nov 4, 2019

Copy link
Copy Markdown
Owner

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants