Thursday, October 8, 2015

Comprehensive LMeve install.


** UPDATE**
July 2016 - If you are trying to upgrade an existing LMeve installation, see my article here: http://myevecorner.blogspot.ca/2016/07/eveonline-upgrading-lmeve.html
I may need to review again this article, due to some modifications in LMeve-0.1.57.

January 2016 : Just noticed that Lukas posted a link to this article. Thanks for that great tool :-)
correction of a few typos
YAML import bug, I found why it wasn't working for me, that was due to the default PHP config.

**

_ Introduction.

For the sake of writing this guide, I'm using a brand new Virtual Machine running CentOS 7, but you could choose whatever Unix you are the most used to use. Everything is pretty standard to be able to use LMeve. You need an Apache, PHP, PHPMyadmin, mysqld and a few libraries.

I decided to use CentOS since it's one of the most used Linux distribution, and I had to do some @work related research on it.
Bear in mind that I will disable some security features on it, since that installation will be completely unreachable from Internet, and thus, invulnerable from attacks from the outside world. If you want to have your LMeve instance accessible for your corporation/alliance, you will have to slightly change the network setup, and reactivate the firewall/SElinux, or equivalent on your unix machine, with the appropriate rules and settings.

On this guide, I do focus only on the installation part of LMeve. If you need help on installing Linux, or configuring it, you won't find it here. I assume that if you use unix, you are, at least, familiar with the basic concepts at managing it.
All commands are run as root, since I'm an old school guy, I don't use sudo at all.

_ Pre-requisites.

Nothing much, safe a linux VM and a few downloads from CCP, from Lukas' site and some files
from Steve Ronuken's site: 
 Aegis_1.2_114300_db.zip (https://developers.eveonline.com/resource/static-data-export)
 lmeve-0.1.53-beta.tar.gz (https://github.com/roxlukas/lmeve/releases)
 yaml_eve_aeg100_dbo.sql.zip (https://pozniak.pl/lmevedbdump/)
 sde from https://www.fuzzwork.co.uk/dump/  (*)
 Aegis_1.2_Icons.zip      (https://developers.eveonline.com/resource/image-export-collection)
 Aegis_1.2_Types.zip

  (*) For Aegis, mysql-latest.tar.bz + invCategories.sql.bz2 are needed, due to a bug at the Aegis release.

Those files are for the Aegis release in July 2015.You will have to adapt those filenames.
LMeve has evolved since I wrote that guide too, download the latest release from GitHUB.


_ First thing first.
Since my VM will be unreachable from Internet, I don't need the firewall nor the SElinux, so, I stop and disable them for good. You will need to revisit this if your instance is to be available for your corp/alliance.

# systemctl status firewalld
# systemctl disable firewalld
# systemctl stop firewalld
# systemctl status firewalld

Now, disable SELinux (or use permissive)
# sestatus
# vi /etc/selinux/config
                change SELINUX to "disabled".
# reboot


_ Installing LMeve dependencies.

Let's get a few tools and repositories (these are for CentOS/Redhat Linux 7):

# yum install wget
# wget http://fedora-epel.mirror.iweb.com//epel-release-latest-7.noarch.rpm
# rpm -ivh epel-release-latest-7.noarch.rpm
# wget http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el7.rf.x86_64.rpm
# rpm -Uvh rpmforge-release-0.5.3-1.el7.rf.x86_64.rpm
# wget http://dev.mysql.com/downloads/file.php?id=450705 (ou lynx)
# rpm -Uvh mysql-community-release-el7-5.noarch.rpm

CentOS doesn’t provide a ftpd. Since I needed one to upload LMeve files, I had to install one, you may skip this step if you could get LMeve and CCP files directly onto the linux.
# yum install vsftpd.x86_64
# cd /etc/vsftpd/
# cp vsftpd.conf vsftpd.conf.orig
# vi vsftpd.conf               
                edit anonymous_enable to No, local_enable=YES, write_enable=YES.
                Check the listen= versus listen_IPV6=, choose to suits your setup.
# systemctl restart vsftpd
# systemctl enable vsftpd

Since my linux setup is a basic one, I have to add a few more tools, you may or may not need them. Your linux distro may have them already installed.

# yum install mysql-community-server.x86_64
# mysql_secure_installation

# yum install httpd
# yum install php-devel
# yum install libyaml-devel
# yum install php-pear
# yum install php-mysql.x86_64

# yum install phpmyadmin
# cd /etc/httpd/conf.d/
# cp phpMyAdmin.conf phpMyAdmin.conf.orig
# vi phpMyAdmin.conf               
                add Require ip and Allow ip for your own IP address
# pecl install yaml

** update January 2016 **  YAML import fix.
# vi /etc/php.ini 
     add "extension=yaml.so" in that file.
     Search for "memory_limit" and make sure the value is at least 1024M.
     ( On CentOS/Redhat, that default value is 128M.)
        

# vi /etc/httpd/config/httpd.conf
    Add somewhere in that file:

                Alias /lmeve /opt/lmeve/wwwroot
                <Directory /opt/lmeve/wwwroot>
                    Order allow,deny
                    Allow from all
                    #uncomment the line below in Apache 2.4; for Apache 2.2 leave it as it is
                    #Require all granted
                    Options FollowSymLinks
                </Directory>

Let's start apache, and connect to http://xx.xx.xx.xx/ to see if Apache has started as it's supposed to. If you don't see the test page from Apache, review your httpd.conf, you may have a typo or a misconfiguration.


_ Install and configure LMeve.

Due to my network configuration, I had to upload LMeve/CCP files via ftp to a local account
on that VM (named "user"). Replace that "~user" by your own path.

#  cd /opt
# tar xzvf ~user/lmeve-0.1.53-beta.tar.gz
# ln -s lmeve-0.1.53-beta lmeve
# cd /opt/lmeve/config
# cp config-dist.php config.php
# vi config.php
                edit:   $LM_EVEDB='eve_tia100_dbo';  .. replace by the current eve db name (eve_aeg100_dbo in this installation)
                               $LM_SALT='abcde12345';            .. create a random chain (hint: select a part of an encrypted password in /etc/shadow)

Let's create our 2 databases. One will be for LMeve internal stuff, the other is the
CCP's one. You could use whatever you want for CCP's, but I decided to stick with CCP's
naming system. We will have to create a mysql user named "lmeve" with password "password".

** update january 2016 ** sticking with CCP's DB name may not be the wisest choice. If I had to restart everything, I would choose a more generic one. eve_ccp_dbo. I don't think that we need to keep older release of CCP's schemas

# mysql -u root -p           (use the root password you created while setting up the mysql.)
                mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
+--------------------+
3 rows in set (0.00 sec)

                mysql> create database lmeve;
                mysql> create database eve_aeg100_dbo;
                mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| eve_aeg100_dbo     |
| lmeve              |
| mysql              |
| performance_schema |
+--------------------+
5 rows in set (0.00 sec)

                mysql> create user 'lmeve'@'localhost' identified by 'password';
                mysql> grant all privileges on * . * to 'lmeve'@'localhost';
                mysql> flush privileges;
                mysql> quit

Now, inject LMeve schemas in the LMeve database.
# cd /opt/lmeve/data
# mysql -u root -p lmeve < schema.sql

And create the admin/admin login/passwd within LMeve.
# cd /opt/lmeve/data
# php ../bin/passwd.php

# Let's add Icons and Types ID from CCP.
# mkdir -p /opt/STATIC_FROM_CCP
# cd /opt/STATIC_FROM_CCP
# unzip ~user/Aegis_1.2_Icons.zip
# unzip ~user/Aegis_1.2_Types.zip
# cd /opt/lmeve/wwwroot
# rm -fr  ccp_icons ccp_img
# ln -s /opt/STATIC_FROM_CCP/Icons/items/ ccp_icons
# ln -s /opt/STATIC_FROM_CCP/Types ccp_img

We're almost done. We need to get the Static Dump from CCP DB. For that, we use Steve Ronuken's work at reformatting that static dump to various DB format. For Aegis, CCP introduced some DB reworks, that's why we have 2 files to import:

# cd ~user
# mkdir fuzzywork_kdump
# cd fuzzywork_kdump/
# bunzip2 -c ../mysql-latest.tar.bz2 | tar xvf -
# cp ../invCategories.sql.bz2 .
# bunzip2 invCategories.sql.bz
# cd aegis-1.1.1-114255/
# mysql -u root -p eve_aeg100_dbo < mysql56-aegis-1.1.1-114255.sql
# cd ..
# mysql -u root -p eve_aeg100_dbo < invCategories.sql
# cd /opt/lmeve/data/
# mkdir eve_aeg100_dbo
# cd eve_aeg100_dbo
# ln -s ~user/fuzzywork_kdump/aegis-1.1.1-114255/*.yaml .

(I did a link to the extracted files, but you could move or copy them instead, and then clean up the ~user home directory.)



On the next command, it failed for me with LMeve 0.1.53. It may be working for you, or the process may change for future release, since it was a brand new process in the 0.1.53.

*** Failed to import YAML.. I'm not a PHP expert, I didn't investigate why.
# php /opt/lmeve/bin/update_yaml.php --all                       
LMeve YAML static data updater
(c) 2014 by Lukasz "Lukas Rox" Pozniak

Using static data from ./data/eve_aeg100_dbo/

Calling updateYamlTypeIDs(), please wait...
loading YAML...PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 520 bytes) in /opt/lmeve-0.1.53-beta/include/yaml_common.php on line 5
#
** Update january 2016 **
As I found the issue while updating my LMeve install in december, I updated this procedure. Make sure that you have changed the "memory_limit" parameter in /etc/PHP.ini to, at least, 1024M. The default value was 128M for me, that's why the import was failing.

# php /opt/lmeve/bin/update_yaml.php --all                       
LMeve YAML static data updater
(c) 2014 by Lukasz "Lukas Rox" Pozniak

Using static data from ./data/eve_aeg100_dbo/

Calling updateYamlTypeIDs(), please wait...
... etc..

If the script fail to import the YAML files, here's the manual import:

Hopefully, Lukas has provided us with a dump of those files, you could download it here: https://pozniak.pl/lmevedbdump/yaml_eve_aeg100_dbo.sql.zip (or a newer version of this file)

*** using pre-installed YAML from lmeve.

# cd ~user
# mkdir YAML_extract
# cd YAML_extract
# unzip ../yaml_eve_aeg100_dbo.sql.zip
# mysql -u root -p eve_aeg100_dbo < eve_aeg100_dbo.sql

_ The final steps

Use the CCP web site to create a corporate API Key, and inject it in the LMeve DB:
# mysql -u root -p lmeve
                mysql> insert into cfgapikeys values (1,'api_key_id','api_verification_code');
                mysql> quit

Edit the following file to change to your timezone:
                poller.php:date_default_timezone_set('Europe/Warsaw');

Change "Europe/Warsaw" to whatever your timezone is. If you don't know its value, check is there:
# ls -la /etc/localtime
lrwxrwxrwx. 1 root root 38 Jul  7 15:58 /etc/localtime -> ../usr/share/zoneinfo/America/New_York
 For me, it’s America/New_York

Finaly, the last touch:
# echo "*/15 * * * * /usr/bin/php /opt/lmeve/bin/poller.php" | crontab
# crontab -l
*/15 * * * * /usr/bin/php /opt/lmeve/bin/poller.php

That last one tell the server to execute the poller at 0, 15, 30 et 45 every hour. The poller has a timeout which prevent it from exceeding 15 minutes of runing time, and thus, you should never see two poller.php running at the same time.

On your workstation (or on the VM if you have installed a graphical interface and a web browser), use your favorite web browser, and go to http://xx.xx.xx.xx/lmeve, login as "admin" with the password "admin" (without "), and in the next 15 minutes, LMeve will come to life.

Congratulation, you just installed LMeve successfully.


3 comments:

  1. Hi,

    Excellent guide. One thing changed in October patch, you no longer have to inject api keys via SQL. There is now a GUI for API Keys: http://pozniak.pl/wp/?p=9577

    ReplyDelete
  2. Hi Lukas, thanks for passing by :-)
    Oh, that's a nice addition to deal with the corp key.
    As soon as you release a new version, I will publish a new install guide, and an upgrade one.

    Keep up the good work!

    ReplyDelete
  3. is there any manual on how to set it up on windows?

    ReplyDelete