Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] launcher issue using jupyterlab 3.6.3 / sparkmagic 0.21.0 #866

Open
achovelon opened this issue Jan 2, 2024 · 5 comments
Open

[BUG] launcher issue using jupyterlab 3.6.3 / sparkmagic 0.21.0 #866

achovelon opened this issue Jan 2, 2024 · 5 comments
Labels
kind:bug An unexpected error or issue with sparkmagic

Comments

@achovelon
Copy link

Describe the bug
Since we migrated jupyter old-notebook version to jupyterlab 3.6.3, we have interface issue when launching sparkmagic. It's working fine and we can create spark session, but we have displayed 2 launchers.

I suspect a compatibility issue with other componant but I couldn't found which one. Is it something you have already experienced ?

To Reproduce
We launch sparkmagic using this command :

%load_ext sparkmagic.magics
%manage_spark

Expected behavior
On the old jupyter the display was fine :
image

And we were using sparkmagic 0.19.1 and jupyter notebook 6.4.6

Screenshots
image

Versions:
$ jupyter --version
Selected Jupyter core packages...
IPython : 8.13.2
ipykernel : 6.23.0
ipywidgets : 8.0.6
jupyter_client : 8.2.0
jupyter_core : 5.3.0
jupyter_server : 2.5.0
jupyterlab : 3.6.3
nbclient : 0.7.4
nbconvert : 7.4.0
nbformat : 5.8.0
notebook : 6.5.4
qtconsole : 5.5.1
traitlets : 5.9.0

  • SparkMagic : 0.21.0
  • Livy (if you know it) : 0.8.0-incubating_2.12 (scala 2.12)
  • Spark : 3.2.4
Copy link

dosubot bot commented Jan 2, 2024

🤖

Thank you for providing detailed information about the issue you're experiencing. To better understand the problem, could you please provide the following additional details:

  1. Are there any error messages in the JupyterLab console when you see the double launcher issue?
  2. Could you provide the list of other JupyterLab extensions you have installed? You can get this list by running jupyter labextension list in your terminal.
  3. Have you tried to reproduce this issue in a clean environment with only JupyterLab and Sparkmagic installed?

These details will help us to better understand the issue and find a solution.


This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

@dosubot dosubot bot added the kind:bug An unexpected error or issue with sparkmagic label Jan 2, 2024
@devstein
Copy link
Collaborator

devstein commented Jan 4, 2024

Hey @achovelon thanks for making an issue!

In the screenshots, it shows you have a PySpark kernel, which means your shouldn't need those line magics in the first celll

%load_ext sparkmagic.magics
%manage_spark

Can you try it without them and see if everything looks good?

Note: Dosu (the bot) can't read screenshots so ignore it

@achovelon
Copy link
Author

Hello,
Thanks for your message. Indeed it works correctly - with or without these two lines.

My point is that after I loaded sparkmagic, when I use %manage_spark it displays in duplicates with the latest version 0.21.0 and it was not the case with the 0.19.1. But I also upgraded jupyter notebook to jupyterhub.

This does not affect our work with Jupyter and Sparkmagic but I wanted to report the display problem. I suspect incompatible versions but I have done a lot of testing and I always come across duplicates displayed using the new version.
Do you have any recommandation on the jupyterhub version to use with sparkmagic 0.21.0 ?

If it's something you couldn't reproduce using jupyterhub 3.2.3 and sparkmagic 0.21.0 I will close the issue.

Thanks for your time.

@devstein
Copy link
Collaborator

@achovelon I see. Thank you for flagging. I'll see if we can reproduce and fix

@jshmchenxi
Copy link

We've encountered similar issue with jupyterlab 4.2.0 / sparkmagic 0.21.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:bug An unexpected error or issue with sparkmagic
Projects
None yet
Development

No branches or pull requests

3 participants