docker기반 데이터 시각화툴 Superset 설치하기 (리눅스)

설치환경 : 리눅스 centos7버전 때

 

기본 환경설정(git)

추후 superset git repository clone을 위해 필요

sudo yum install git

docker 설치 centos7

참고문서 : https://linuxize.com/post/how-to-install-and-use-docker-on-centos-7/

1. Start by updating your system packages and install the required dependencies:

sudo yum update
sudo yum install yum-utils device-mapper-persistent-data lvm2

2. Next, run the following command which will add the Docker stable repository to your system:

sudo yum-config-manager --add-repohttps://download.docker.com/linux/centos/docker-ce.repo

3. Now that the Docker repository is enabled, install the latest version of Docker CE (Community Edition) using yum by typing:

sudo yum install docker-ce

4. Once the Docker package is installed, start the Docker daemon and enable it to automatically start at boot time:

sudo systemctl start docker sudo systemctl enable docker

5. To verify that the Docker service is running type:

sudo systemctl status docker

 

Executing the Docker Command Without Sudo #

By default managing, Docker requires administrator privileges. If you want to run Docker commands as a non-root user without prepending sudo you need to add your user to the docker group which is created during the installation of the Docker CE package. You can do that by typing:

 

sudo usermod -aG docker ${USER}

$USER is an environment variable that holds your username.

 

Docker Compose 설명 및 설치

1. docker-compose download

sudo curl -L "https://github.com/docker/compose/releases/download/1.23.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

2. APPLY EXECUTABLE PERMISSION TO THE BINARY
sudo chmod +x /usr/local/bin/docker-compose

 

3. alias setting

sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

docker, docker-compose 설치 이후 compose실행시 에러가 난다면 해당 세션창을 나갔다가 다시 접속해야한다.

superset source code clone & install

1. clone superset source codegit clone https://github.com/apache/incubator-superset/

 

2. dockerfile 수정

# First, we just wanna install requirements, which will allow us to utilize the cache

# in order to only build if and only if requirements change

COPY ./requirements.txt /app/ RUN cd /app \         && pip install --no-cache -r requirements.txt

해당 부분에 pip로 사용할 라이브러리들 추가

COPY ./requirements.txt /app/

RUN cd /app \        

      pip install --upgrade pip \        

      && pip install --no-cache -r requirements.txt \        

      && pip install mysqlclient \        

      && pip install impyla \        

      && pip install redis==3.2.1 \        

      && pip install gevent==1.2.2 \        

      && rm -rf /root/.cache/pip

 

 

3. docker-compose.yml 파일 수정

version: "3.7" => version: "3.4"
  superset:
    ports:
      - 8088:8088  => -8089:8088     (8088포트로 띄울 경우 정상적으로 동작안함)

4. docker-compose up

docker-compose up 명령어로 dockerfile의 이미지를 읽어가며 설치

docker-compose up

기존 리서치 superset에서 발생했던 이슈

1. 다운로드한 CSV가 엑셀로 열 경우 한글이 깨지는 문제 수정

2. sql tab이 한글명일 경우 정상적으로 csv다운로드 안되는 이슈 처리

vi /home1/irteam/incubator-superset/superset/views/core.py

 

docker기반 superset 설치시 발생했던 이슈 정리

1.설치는 되었는데 default id/pw가 생성이 안되어 있을 경우 =>  docker-init을 통하여 계정 설정

docker-init

## Initializing Database To initialize the database with a user and example charts, dashboards and datasets run:

```bash

docker-compose run -e SUPERSET_LOAD_EXAMPLES=yes --rm superset ./docker-init.sh

 

2. docker-compose.yml 내부의 버전과 docker-compose 버전이 맞지 않는 경우

[incubator-superset]$ docker-compose up



ERROR: In file './docker-compose.yml' service 'version' doesn't have any configuration options. All top level keys in your docker-compose.yml must map to a dictionary of configuration options.

해결책 docker-compose version 3.7 -> 2버전대로 낮추거나 docker-compose 버전을 높여주어야 한다.
참고 : https://github.com/docker/compose/issues/3331

 

3. 알파환경에서는 문제가 없었는데 리얼환경에서 클러스터 OS Partition 차이로 인한 이슈

-> docker-compose up 명령어 실행시 발생

ERROR: Service 'superset' failed to build: OCI runtime create failed: /var/lib/docker/overlay2/17f563586397ce35fab5733d85b37fc68073a42eb4a5044025e68fd89e87b573/merged is not an absolute path or is a symlink

해결책

# 실행되고 있는 docker stop
$ sudo systemctl stop docker

# 잘죽었는지 확인
$ sudo systemctl status docker

# 원하는 디렉터리 이동을 위한 디렉터리 생성
$ sudo mkdir /mnt/extra

# 기존 문제 발생했던 디렉터리의 내용들을 새로 만든 디렉터리로 이동
$ sudo mv /var/lib/docker /mnt/extra

# 링크생성
$ sudo ln -s /mnt/extra/docker /var/lib/docker

# docker 시작
$ sudo systemctl start docker

# docker 잘올라왔는지 상태 확인
$ sudo systemctl status docker

# 재설치 (iteam) $ docker-compose up