Category Archives: Linux

Linux Category

Monitor Temperature and Humidity using Raspberry Pi

Today I would like to share you the way to monitor via web page temperature and humidity using raspberry Pi and Sensor for humidity and temperature. You could find a lot of similar articles in the Internet. Mine solution is just combination of several and small adjustment.

Level of moisture in our house is too high and this requires intervention from our side to reduce it. Recommended value of moisture should be between 35% and 55%. High level of moisture increase probability to get ingress on our things, clothes, walls..

Moisture level depends on air flows and temperature in your house. If in the winter temperature is below 20 Celsius degree the level of humidity will be higher than recommended value. Minimum recommended level of temperature should be 21 Celsius degrees.  In other words you need to have smart heating system and good ventilation to reduce fast humidity when you are cooking or after you used bathroom. So let’s see how to get data which we need for decisions.

We need equipment:

  • DHT22 AM2302 Digital Temperature and Humidity Sensor module Replace SHT11 SHT15, Price – 5.97 USD
  • Raspberry Pi-2 + Acrylic Enclosure Case + Heat sink, Price – 42.27 USD 

Let’s connect sensor to the raspberry. I did everything like in this link http://www.home-automation-community.com/temperature-and-humidity-from-am2302-dht22-sensor-displayed-as-chart/ except the thing that I didn’t used resistor and development plate. My sensor has resistor built in. Here is my equipment and connections:

Before to collect data let’s update our raspberry:

sudo aptget update

Note: Many steps in this tutorial require internet acces on raspberry PI (If you don't know how to do it see my previous article how to connect raspbery to the internet via Wi-Fi)

Raspberry collect data from sensor on the GPIO pin 4. To read data we will use python library called Adafruit_Python_DHT.

To install library, get the dependencies with:

sudo aptget install y buildessential pythondev git

and then download and install the library with:

mkdir -p /home/pi/sources 
cd /home/pi/sources 

git clone https://github.com/adafruit/Adafruit_Python_DHT.git 

cd Adafruit_Python_DHT 

sudo python setup.py install 

 Here we can do fast check to see that Pi is ready to collect data from sensor:

sudo /home/pi/sources/Adafruit_Python_DHT/examples/AdafruitDHT.py 2302 4

The first argument is the sensor type, it can be 11 or 22 or 2302. The second argument is the RPi GPIO pin which is connected to the sensor data pin.

Output will be like:

sudo /home/pi/sources/Adafruit_Python_DHT/examples/AdafruitDHT.py 2302 4

Temp = 24.2*C Humidity=26.3%

Let’s create sql database to collect our data

Install sqlite

sudo apt-get install sqlite3

  Create DB schema

BEGIN;

CREATE TABLE temp_hum (timestamp DATETIME, temp NUMERIC, hum NUMERIC);

COMMIT;

Great, now we have DB where we could write data. We created table called temp_hum. This table has 3 columns: timestamp, temp and hum. First column will contain actual date and time. So let’s assure that time and data is right. Use date command to check it.

If something is wrong let’s install ntpdate packet and adjust time:

sudo apt-get install ntpdate

sudo service ntpd stop

sudo ntpdate time.nist.gov

sudo service ntpd start

Last step is to adjust timezone. For me timezone is GMT+3. Find your on the internet if you don’t know which one is yours.

sudo cp /usr/share/zoneinfo/Etc/GMT+3 /etc/timezone

Let’s install and configure web server

sudo apt-get install apache2

We will use cgi-bin scripts to display collected data on web page. So let's say apache web server to use them. This step could be optional for you:

a2enmod cgi

a2enmod cgid

service apache2 restart

Move created DB to the /var/www/ folder and change owner of DB:

sudo cp templog.db /var/www/
sudo chown www-data:www-data /var/www/templog.db 

Now we have to do little bit programming or just copy ready solution from the internet (as I did, thanks this author for help). I adjusted liitle bit found code in the internet in order to get expected results.

nano /root/monitor.py

and insert following text inside:

#!/usr/bin/env python

import sqlite3
import Adafruit_DHT
import os
import time
import glob

# global variables
dbname='/var/www/templog.db'

# store the temperature and hummidity in the database
def log_temperature(temp,humm):

    conn=sqlite3.connect(dbname)
    curs=conn.cursor()

    curs.execute("INSERT INTO temp_hum values(datetime('now'), (?), (?))", (temp,humm))
    #curs.execute("INSERT INTO temp_hum values(datetime('now'), (?))", (humm,))
    # commit the changes
    conn.commit()

    conn.close()


# display the contents of the database
def display_data():

    conn=sqlite3.connect(dbname)
    curs=conn.cursor()

    for row in curs.execute("SELECT * FROM temp_hum"):
        print str(row[0])+”     “+str(row[1])+”        “+str(row[2])

    conn.close()

# get temerature and humidity
# returns None on error, or the temperature,humidity as a float
def get_temp_hum():
        try:
                humm, temp = Adafruit_DHT.read_retry(Adafruit_DHT.DHT22, 4)
                humm = round (humm, 2)
                temp = round (temp, 2)
                return humm, temp
        except:
                return None, None

# main function
# This is where the program starts
def main():
        # get the temperature from the device file
        humm, temp = get_temp_hum()
        # Store data to DB in case we have values
        if (humm != None and temp !='None'):
                log_temperature(humm, temp)
#       display_data()

if __name__=="__main__":
    main()

Let me add several comments to the code. First of all we defined database location dbname='/var/www/templog.db'. After that we defined function log_temperature to insert data in our DB. We have function called get_temp_hum to read data from the sensor using python library installed at the beginning of this article.The most important part is main function where we are calling get_temp_hum, check that we have valid data and insert them to DB using log_temperature. One more function display_data() is commented and we need it only for debug purposes. Uncomment this line and execute the script to assure that you have inserted data in the DB.

Don't forget to make file monitor.py executable:

chmod 755 /root/monitor.py

Let's collect data every 15 minutes. For this print:

crontab -e

Insert this line:

*/15 * * * * /root/monitor.py

Note: If execution of script fail for some reason try to excute script manually with sudo /root/monitor.py. Also you could monitor script execution in crontab using postfix mail service. To install it use sudo apt-get install postfix.

Last step is create script which will display data on the web page.

nano /usr/lib/cgi-bin/webgui.py

Insert there following text:

#!/usr/bin/env python
import sqlite3
import os
import sys
import cgi
import cgitb

# global variables
dbname='/var/www/templog.db'

# print the HTTP header
def printHTTPheader():
    print "Content-type: text/html\n\n"

# print the HTML head section
# arguments are the page title and the table for the chart
def printHTMLHead(title, table):
    print "<head>"
    print "    <title>"
    print title
    print "    </title>"

    print_graph_script(table)

    print "</head>"


# get data from the database
# if an interval is passed,
# return a list of records from the database
def get_data(interval):

    conn=sqlite3.connect(dbname)
    curs=conn.cursor()

    if interval == None:
        curs.execute("SELECT * FROM temp_hum")
    else:
        curs.execute("SELECT * FROM temp_hum WHERE timestamp>datetime('now','-%s hours') AND timestamp<=datetime('now')" % interval)

    rows=curs.fetchall()

    conn.close()
    return rows


# convert rows from database into a javascript table
def create_table(rows):
    chart_table=""

    for row in rows[:-1]:
        rowstr="[‘{0}’, {1}, {2}],\n”.format(str(row[0]),str(row[1]),str(row[2]))
        chart_table+=rowstr

    row=rows[-1]
    rowstr="[‘{0}’, {1}, {2}]\n”.format(str(row[0]),str(row[1]),str(row[2]))
    chart_table+=rowstr

    return chart_table


# print the javascript to generate the chart
# pass the table generated from the database info
def print_graph_script(table):

    # google chart snippet
    chart_code="""
    <script type="text/javascript" src="https://www.google.com/jsapi"></script>
    <script type="text/javascript">
      google.load("visualization", "1", {packages:[“corechart”]});
      google.setOnLoadCallback(drawChart);
      function drawChart() {
        var data = google.visualization.arrayToDataTable([
          [‘Time’, ‘Humidity’, ‘Temperature’],
%s
        ]);
        var options = {
          title: 'Temperature and Humidity'
        };
        var chart = new google.visualization.LineChart(document.getElementById('chart_div'));
        chart.draw(data, options);
      }
    </script>"""

    print chart_code % (table)

 


# print the div that contains the graph
def show_graph():
    print "<h2>Temperature Chart</h2>"
    print '<div id="chart_div" style="width: 900px; height: 500px;"></div>'

# connect to the db and show some stats
# argument option is the number of hours
def show_stats(option):

    conn=sqlite3.connect(dbname)
    curs=conn.cursor()

    if option is None:
        option = str(24)

    curs.execute("SELECT timestamp,max(temp) FROM temp_hum WHERE timestamp>datetime('now','-%s hour') AND timestamp<=datetime('now')" % option)
    tempmax=curs.fetchone()
    tempmax="{0}&nbsp&nbsp&nbsp{1} %".format(str(tempmax[0]),str(tempmax[1]))

    curs.execute("SELECT timestamp,min(temp) FROM temp_hum WHERE timestamp>datetime('now','-%s hour') AND timestamp<=datetime('now')" % option)
    tempmin=curs.fetchone()
    tempmin="{0}&nbsp&nbsp&nbsp{1} %".format(str(tempmin[0]),str(tempmin[1]))

    curs.execute("SELECT avg(temp) FROM temp_hum WHERE timestamp>datetime('now','-%s hour') AND timestamp<=datetime('now')" % option)
    tempavg=curs.fetchone()

    curs.execute("SELECT timestamp,max(humm) FROM temp_hum WHERE timestamp>datetime('now','-%s hour') AND timestamp<=datetime('now')" % option)
    hummmax=curs.fetchone()
    hummmax="{0}&nbsp&nbsp&nbsp{1} C".format(str(hummmax[0]),str(hummmax[1]))

    curs.execute("SELECT timestamp,min(humm) FROM temp_hum WHERE timestamp>datetime('now','-%s hour') AND timestamp<=datetime('now')" % option)
    hummmin=curs.fetchone()
    hummmin="{0}&nbsp&nbsp&nbsp{1} C".format(str(hummmin[0]),str(hummmin[1]))

    curs.execute("SELECT avg(humm) FROM temp_hum WHERE timestamp>datetime('now','-%s hour') AND timestamp<=datetime('now')" % option)
    hummavg=curs.fetchone()


    print "<hr>"


    print "<h2>Minumum Value&nbsp</h2>"
    print tempmin
    print hummmin
    print "<h2>Maximum Value</h2>"
    print tempmax
    print hummmax
    print "<h2>Average Value</h2>"
    print "%.1f" % tempavg+"%"
    print "%.1f" % hummavg+"C"
    print "<hr>"

    print "<h2>In the last hour:</h2>"
    print "<table>"
    print "<tr><td><strong>Date/Time</strong></td><td><strong>Temperature</strong><td><strong>Humidity</strong></td></td></tr>"

    rows=curs.execute("SELECT * FROM temp_hum WHERE timestamp>datetime('now','-1 hour') AND timestamp<=datetime('now')")
    for row in rows:
        tempstr="<tr><td>{0}&emsp;&emsp;</td><td>{1} %</td><td>{2} C</td></tr>".format(str(row[0]),str(row[1]),str(row[2]))
        print tempstr
    print "</table>"

    print "<hr>"

    conn.close()

def print_time_selector(option):

    print """<form action="/cgi-bin/webgui.py" method="POST">
        Show the temperature logs for
        <select name="timeinterval">"""


    if option is not None:

        if option == "6":
            print "<option value=\"6\" selected=\"selected\">the last 6 hours</option>"
        else:
            print "<option value=\"6\">the last 6 hours</option>"

        if option == "12":
            print "<option value=\"12\" selected=\"selected\">the last 12 hours</option>"
        else:
            print "<option value=\"12\">the last 12 hours</option>"

        if option == "24":
            print "<option value=\"24\" selected=\"selected\">the last 24 hours</option>"
        else:
            print "<option value=\"24\">the last 24 hours</option>"

        if option == "168":
            print "<option value=\"168\" selected=\"selected\">the last 168 hours</option>"
        else:
            print "<option value=\"168\">the last 168 hours</option>"

    else:
        print """<option value="6">the last 6 hours</option>
            <option value="12">the last 12 hours</option>
            <option value="24">the last 24 hours</option>
            <option value="168">the last 168 hours</option>"""

    print """        </select>
        <input type="submit" value="Display">
    </form>"""


# check that the option is valid
# and not an SQL injection
def validate_input(option_str):
    # check that the option string represents a number
    if option_str.isalnum():
        # check that the option is within a specific range
        if int(option_str) > 0 and int(option_str) <= 24:
            return option_str
        else:
            return None
    else:
        return None


#return the option passed to the script
def get_option():
    form=cgi.FieldStorage()
    if "timeinterval" in form:
        option = form[“timeinterval”].value
        return validate_input (option)
    else:
        return None

# main function
# This is where the program starts
def main():

    cgitb.enable()

    # get options that may have been passed to this script
    option=get_option()

    if option is None:
        option = str(24)

    # get data from the database
    records=get_data(option)

    # print the HTTP header
    printHTTPheader()

    if len(records) != 0:
        # convert the data into a table
        table=create_table(records)
    else:
        print "No data found"
        return

    # start printing the page
    print "<html>"
    # print the head section including the table
    # used by the javascript for the chart
    printHTMLHead("Raspberry Pi Temperature and Humidity Logger", table)

    # print the page body
    print "<body>"
    print "<h1>Raspberry Pi Temperature and Humidity Logger</h1>"
    print "<hr>"
    print_time_selector(option)
    show_graph()
    show_stats(option)
    print "</body>"
    print "</html>"

    sys.stdout.flush()

if __name__=="__main__":
    main()

Let me again comment the script. First of all we need to know how much hours to display in graph using get_option function. By default 24 hours will be displayed. Function get_data will collect dates from DB. If you have at least one string in the DB data will be converted create_table function to display everything on the web. Actual string is not converting data for 1 week. This is what I have to fix after.

Don't forget to make file webgui.py executable:

chmod 755 /usr/lib/cgi-bin/webgui.py

To see data in the web page open web browser on the PC in the same network with raspberry and insert in the address field:

192.168.1.10/cgi-bin/webgui.py

192.168.1.10 is the ip address of Pi

Final result look like this:

After 10 AM I moved sensor closer to the heating system. This is why we have so big changes in the graph.

Links

  1. http://www.home-automation-community.com/temperature-and-humidity-from-am2302-dht22-sensor-displayed-as-chart/
  2. https://github.com/adafruit/Adafruit_Python_DHT
  3. http://raspberrywebserver.com/cgiscripting/rpi-temperature-logger/building-a-web-user-interface-for-the-temperature-monitor.html

Connect raspberry pi to Wi-FI

Some time ago I decided to become more familiar with Internet of things. I thought about several projects, but I didn't decide what to do this is why I bought several things and forgot about them for a while.

Few days ago I decided to take a look inside. Newbies (like me) think that pieces for the project could be expensive, actually they are not so. I decided to publish what I bought and how much it cost.

  • Electronic Parts Pack KIT for ARDUINO component Resistors Switch Button WT, Price – 3.70 USD
  • Soil Hygrometer Humidity Detection Module Soil Moisture Water Sensor, Price –  1.16 USD
  • MB102 Power Supply Module 3.3V 5V+Breadboard Board 830 Point+65PCS Jumper cable, Price – 5.08 USD
  • DHT22 AM2302 Digital Temperature and Humidity Sensor module Replace SHT11 SHT15, Price – 5.97 USD
  • UNO R3 ATmega328P CH340 Mini USB Board for Compatible-Arduino NEW, Price – 3.95 USD
  • Raspberry Pi-2 + Acrylic Enclosure Case + Heat sink, Price – 42.27 USD 
  • Micro USB Charger 5V,  2A, Price – 7.64 USD
  • Temperature sensor DS18b20 in iron case, Price – 1.25 USD
  • Micro SD memory card Class 10 Size 8GB , Price – 5.17 USD
  • Wi-Fi USD Adapter EDUP EP-N8508GS, Price – 5.66 USD

Let's start with something. I went to the https://www.raspberrypi.org/ and found that I have to copy operating system to the SD card and insert SD card to the raspberry. Ok I decided to load NOOBS OS and after that select raspbian OS. I bought Class 10 SD card (fast SD Card ­čÖé ). After I inserted card in the raspberry and connect via HDMI raspberry I found that nothing is printed on screen. First attempt fail, the good idea was to consult here the list of compatible SD card before to buy something. Ok I found at home old SD card at 8 Gb and I decided to try it just for proof of concept. Second attempt was good.

To configure raspberry you have to get USB mouse and Keyboard. Nothing of those I have in the house. After several hours I found only mouse. Ok let's start with this, especially that on first stage I have only to select OS and boot raspberry. After boot I got the screen with GUI and I still don't have keyboard. Probably I could connect via ssh to OS was my idea. I could do it in two ways via LAN or WLAN. My Home router has password, so even wi-fi card is up raspberry doesn't know my password. I connected laptop to device via LAN, bring up on laptop DHCP server (tftpf32 could do it very fast), found which ip address was leased to raspberry and connects to him via putty (default user: pi, password: raspberry). Everything is easy here I thought. Let's connect raspberry to wi-fi. I have never been connect linux to wi-fi before. It's shouldn't be too difficult, especially that I am connecting linux via LAN in a day by day activities. Spent some time on google I found a lot of step-by-step instructions and different options, but no one work for me.

My Wi-Fi Card is:

pi@raspberrypi:~ $ lsusb
Bus 001 Device 004: ID 0bda:8176 Realtek Semiconductor Corp. RTL8188CUS 802.11n WLAN Adapter

My OS is:

pi@raspberrypi:~ $ cat /etc/*release
PRETTY_NAME="Raspbian GNU/Linux 8 (jessie)"
NAME="Raspbian GNU/Linux"
VERSION_ID="8"
VERSION="8 (jessie)"
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

After several hours I start to hate Linux and I saw that I am not single in this feeling. How can so easy task take so much time? I started my attempts with wireless-tools installed on raspbian. Command iwlist wlan0 scan provides info about my router. This means that driver for wlan0 interface was installed and interface is up. Let's try connect manually to router using: iwconfig wlan0 essid <YOUR_SSID> key s:<Your password>.  Several attemps fail. Ok let's try to use another option. I found that wpa_supplicant was created especially for people who want to use Wi-FI in secure way and has WPA2 encryption. Command wpa_passphrase <Your_SSID> <Your_Password> >> /etc/wpa_supplicant/wpa_supplicant.conf will create config file for your Network and save password in the safe way. If select wpa_supplicant method you have to fill /etc/network/interfaces with something like this:

allow-hotplug wlan0
iface wlan0 inet manual
wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf

Inet manual is required by wpa_supplicant even you have to use dhcp. If you want dhcp do this:

allow-hotplug wlan0
iface wlan0 inet manual
wpa-conf /etc/wpa_supplicant/wpa_supplicant.conf

iface default inet dhcp

I created wpa_supplicant config for me, checked /etc/network/interfaces but I still couldn't associate with Wi-Fi. After several attempts I tried to run wpa_supplicant daemoon manually. This part I want to share with you in more details. After network service restart (/etc/init.d/networking restart) in the background is started wpa_supplicant daemoon. Command ps aux | grep wpa* provides me the following:

/sbin/wpa_supplicant -s -B -P /run/wpa_supplicant.wlan0.pid -i wlan0 -D nl80211,wext -c /etc/wpa_supplicant/wpa_supplicant.conf

This still doesn't help me. Somewhere should be the log files. Yes they are in /var/log/syslog or /var/log/messages or /var/log/wpa_supplicant.log. For me it was /var/log/syslog. Inside I found that:

nl80211: Driver does not support authentication/association or connect commands

Hmm, let's kill the process and play manually with different options, drivers of wpa_supplicant.

Sometimes I could get another error:

wlan0: CTRL-EVENT-SSID-TEMP-DISABLED id=0 ssid="Home" auth_failures=11 duration=120 reason=CONN_FAILED

I edited several times wpa_supplicant.conf file with different options and at the end I got something like this:

root@raspberrypi:/home/pi# cat /etc/wpa_supplicant/wpa_supplicant.conf
ap_scan=0
country=GB
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1

network={
        ssid="<Your_SSID>"
        proto=RSN
        scan_ssid=1
        key_mgmt=WPA-PSK
        pairwise=CCMP TKIP
        group=CCMP TKIP
        psk="<Your_Password>"
        #psk=8d0526f31a39e78dfgk;srs61ae5a1e9d0c30e8e48cc838f8541d4a8e7af259 #password generated by wpa_passphrase
}

At the beginning I had only ssid and psk key generated by wpa_passphrase. Information about key_mgmt, pairwise you could get using iwlist wlan0 scan command. Even everything seems to be logic I still couldn’t connect to the Router and get ip address from him. This is why I tried to launch manually supplicant using debug options:

/sbin/wpa_supplicant -s -B -P /run/wpa_supplicant.wlan0.pid -i wlan0 -D nl80211,wext -c /etc/wpa_supplicant/wpa_supplicant.conf -dd

Debug options and /var/log/syslog will give you information what failed for you. Each new option enabled/disabled in wpa_supplicant.conf file and drivers used in command provide new error to my investigation. It's interesting, but didn't help. Try to see in your case this.

Some instructions proposed to change /etc/network/interfaces in different way to connect raspberry to wi-fi. I have new way to investigate. I saw that someone is saying to use wpa-ssid, someone wpa-essid and different things. Why options are different? Let's try all of them :). It's good idea but this still didn't solve the issue. Windows seems to be better ­čÖé was my idea. Trying to find solution for me I found very interesting article written by mrEngman. Inside he shares his script how to configure wi-fi cards based on RTL8188CUS. I decided to read script instead to launch him (thanks author for very good comments inside). And there I found that wpa-ssid is for Router without authentication. For those with autehntication should be wpa-essid. wpa-psk should be used when you have WPA2 (most used), in case you have WEP you shold use wpa-key. Thanks to mrEngman that he decided to explain what means different options. After I edited /etc/network/interfaces like below everything started to work for me.

auto wlan0
iface wlan0 inet dhcp
wpa-essid <Your_SSID>
wpa-psk <Your_Password>

Conclusion: "Never give up!". Linux could be awful at first meet, but when you start to understand and think what is the impact of command, where to see logs, what means your options (use man pages), how to restart process manually and apply changes it becomes logic.

Important: Don't use at the same time iwconfig, wpa_supplicant. Select one method or script and debug it. Remember that processes are running in background and applied commands could affect actual results.

Links:

  1. https://www.raspberrypi.org/ 
  2. http://elinux.org/RPi_SD_cards
  3. http://tftpd32.jounin.net/
  4. http://www.raspberrypi.org/forums/viewtopic.php?t=6256

 

LVM (Logical Volume Manager)

          Deseori studentii chiar din prima lectie de LPI, ma intreaba ce inseamna LVM ­čÖé Si ca sa nu fac copy/paste de pe wiki incerc sa explic in 2 cuvinte sa fie pe intelesul tuturor:

LVM (Logical Volume Manager) – Este un nivel de abstractizare, care ne permite sa folosim 1 sau mai multe discuri rigide (S/HDD) ca o singura entitate, dupa care aceasta entintate o putem partitiona dupa nevoi.

Nota!: Este importat de memorat, ca partitia /boot/ nu poate fi utilizata ca partitie LVM deoarece GRUB nu "intelege" partitiile LVM si respectiv nu poate citi fisierele de configurare si imaginea kernelului, ca urmare nu vom fi capabili sa pornim sistemul de operare.


Spre exemplu avem un JBOD, sau sa luam un exemplu mai simplu – 3 HDD-uri de dimensiuni diferite, fie 10, 20, 30 GB (Total 60Gb). Cu ajutorul tehnologiei LVM unim toate aceste HDD_uri in unul Logic, dupa care il partitionam dupa necesitate.
LVM

Cum lucram cu LVM ?
In LVM avem 3 nivele de abstractizare, si un set de comenzi pentru fiecare nivel care este intuitiv simplu de memorat:

Nivele de abstractizare Comenzi de lucru (lista incompleta)
1. PV (Physical Volume) — Discurile fizice, pot fi atit partitii aparte cat si discurile rigide in intregime.

pvcreate – initializeaza discurile fizice pentru a fi folosite in LVM
pvdisplay – afiseaza informatii despre discurilor fizice
pvremove – sterge un volum fizic

2. VG (Volume Group)  – Grupul de discuri care vor face parte din LVM  (Prin crearea grupei, noi indincam care partitii sau HDD-uri vor face parte din LVM)

vgcreate – creaza grupa de volume
vgdisplay – afiseaza informatii despre grupele de volume
vgreduce – sterge un grup volum

3. LV (Logical Volume) – Partitiile sau partitia logica care va/vor fi create din grupul de volume (VG).

lvcreate – creaza volumul logic
lvdisplay – afiseaza informatii despre volumele logice
lvremove – sterge un volum logic

Trecem la lucru practic:

Pentru a lucra cu LVM avem nevoie de setul de comenzi mentionate anterior, deci instalam pachetul LVM2, in caz ca el deja nu este instalat.

Pentru a vedea ce discuri avem in sistem, ce capacitate, care unde este montat executam lsblk:

Ca rezultat vedem ca la noi in sistem sint 3 HDD-uri sda, sdb si sdc.
sda – care e partitionat in 3, si pe care e instalat sistemul de operare
sdb si sdc – 2 HDD-uri care sint absolut noi, fara oricare file sistem (FS) si partitii.

Scopul nostru este de a crea un LVM din aceste 2 HDD-uri sdb si sdc.

Initializam discul sdb si sdc pentru a fi folosit in LVM:

Acum cream un Volum Group (discul virtual care este format din mai multe partitii sau hdd-uri) cu un nume user friendly

(!) Atrageti atentia ca dupa comenzile pvcreate si vgcreate am indicat doar discurile care vor face parte din LVM

Dupa ce am creat grupul de volume care vor face parte din LVM ne-a mai ramas doar sa cream volumele sau partitiile logice LV (Logical Volumes) :

Sa presupunem ca vrem sa cream 3 partitii logice (Total avem 2Gb, HDD1 + HDD2 = 1Gb +1Gb)
1 = 1.5Gb
2 = 128Mb
3 = 384Mb

Unde:
-n – este optiunea pentru numele partitiei
-L sau –size  – este optiunea pentru marimea partitiei.
MyVG – este numele Grupei de volume

Partitiile (Volumele logice) create vor aparea in /dev/[nume_vg]/ , care de fapt sint niste symlink-uri user friendly

 

Pasul urmator consta in formatare partitiilor sub un anumit file sistem, sau chiar si swap.

ulterior aceste partii pot fi montate si utilizate.

Scopul a fost atins! Ca rezultat am obtinut un disc Logic (Volume Group) din 2 discuri a cate 1Gb, care ulterior a fost repartitionat in 3 partitii.


Acum ca presupunem ca peste ceva timp am mai procurat un HDD (de 4Gb, acest HDD fiind deja al 4-a dupa numar)  care la fel dorim sa-l includem in LVM-ul deja existen. Ca urmare in loc de 2Gb vom avea 6Gb. Deci adaugam noul disc in LVM-ul deja existent:

Dupa cum vedem, hdd-ul a fost adaugat cu success in LVM, acum verificam:

 

In caz ca dorim sa excludem un HDD din LVM  parcurgem pasii urmatori:


Administrare GUI, pentru cei mai putini harnici:
Ca alternativa de administrare a LVM puteti folosi system-config-lvm, care desigur trebuie instalata in sistem

lvextend_ubuntu_root_visual

Sper ca acest articol a adus ceva lumina la capitolul LVM

Enjoy

GNS3 – doesn’t work ping through the cloud on VM

First of all I would like to mention that all actions published here have as scope educational purpose. A lot of people who are trying to use GNS3 for testing/lab purpose meet the problem with connection to Internet through cloud.

My problem was: I couldn't ping from router in gns3 any external host. I meet this problem several times on different Operating Systems with different versions of GNS3. I researched on Internet for the solution many times and every time gave up in front of the problem. This is why I decided to write this article. It could be useful for someone in the same situation.

Let's begin with topology:

  1. I decided to install GNS3 on the Linux Ubuntu 14.04. Big Thanks to people who wrote this article http://www.computingforgeeks.com/2014/12/best-way-to-install-gns3-version-12-in.html and special thanks for those who wrote installation script.
  2. One of the most important things is that I decided to install GNS3 on VM using as hypervisor ESXi. Below you can see the topology. So I selected VM1 for this purpose.
  3. I connected ESXi host to the Cisco's switch using two cables. Cable connected to the G0/1 interface was set to trunk. Cable on G0/2 wasn't used (we will use it for debug purpose later).
  4. My VM1 has two interfaces. One is for remote management and the second for testing purpose.
  5. From VM1 I could ping from VLAN 10 interface User PC.

VMware_GNS3

Let's move to GNS3 and create topology:

I am not going explain how to create topology and connect to the cloud you could easy find how to do it on internet. I will just publish screen from my topology:

VMware_GNS3_topologyAfter topology creation I was configured Interface Fa0/0 to get ip address via DHCP:

R1(config)#int fa0/0
R1(config-if)#ip add dhcp
R1(config-if)#no sh
R1(config-if)#
*Mar  1 00:03:35.071: %LINK-3-UPDOWN: Interface FastEthernet0/0, changed state to up
*Mar  1 00:03:36.071: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/0, changed state to up
R1(config-if)#
*Mar  1 00:03:48.367: %DHCP-6-ADDRESS_ASSIGN: Interface FastEthernet0/0 assigned DHCP address 192.168.140.65, mask 255.255.255.0, hostname R1

Ok I have confirmation that packets are going through cloud. Everything is good. Next test is to ping CR2 interface ip address:

R1(config-if)#do ping 192.168.140.254

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.140.254, timeout is 2 seconds:
…..
Success rate is 0 percent (0/5)

 Hmm.. Maybe ICMP packets were suppressed. Let's try to send ping from VM – ping test passed. So we have the situation when some packets could reach GNS3 and some others couldn't. Let's try to find where is the exactly the problem. Packets could be stuck on VM or somewhere in the network. We are connected to the Cisco switch and we have option to configure SPAN or port mirroring in order to check out suppositions. Let's do it.

C2960(config)#monitor session 1 source interface gigabitEthernet 0/1 both

C2960(config)#monitor session 1 destination interface GigabitEthernet 0/2

Please keep in mind that Gi0/2 interface couldn't send legacy user traffic after this configuration. In our case this is not a problem because I don't use this interface. On the VM2 I run Wireshark application and from R1 in GNS run again ping command. In the wireshark I see that packets leave VM and return back to the host. So problem seems to be somewhere on the VM. It could be SElinux or iptables.

Let's try another thing. Let's check how is set vmnic0 interface for VLAN10 in security tab. For this go the vSphere Client -> Configuration -> Networking -> Properties -> Double-click on your VLAN -> Press Security tab. I had the following configuration:

VMware_GNS3_errorLet's try to change promiscuous mode from Reject to Accept and repeat test again. The results:

R1(config-if)#do ping 192.168.140.254

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.140.254, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/20/44 ms
R1(config-if)#

Here was my problem. Promiscuous mode allows vSwitch to forward all frames including those which are not directed to VM. Router in GNS3 acts as virtual interface inside of VM. From security purpose VMware block frames which are addressed to VM.

I hope that this article could be useful for you.

 

Backup to Dropbox

skachat-dropbox-04     Deseori se intimpla ca avem nevoie sa facem copii de rezerva, fie a bazei de date, fie a anumitor fisiere.
In dependenta de gravitatea pierderii informatiei, fiecare in parte decide cand, unde, cum face copii de rezerva. Varianta ideala este de a pastra 3 copii in locatii geografice diferite, insa fiecare are “planul” sau de backup/recovery.

In acest post, astazi voi descrie cum putem face un backup planificat direct pe Dropbox. Asemanator poate fi creata procedura de backup pe un server S/FTP sau cloud de genul OneDrive sau Google Drive, insa cu ajustarile necesare in dependenta de tipul lagaturii cu serviciile cloud.

Dropbox ofera API pentru accesarea serviciilor sale, ceia ce face comod de a lucra sub linia de comanda direct. Pentru a nu inventa bicicleta, pe internet se gaseste deja un script care permite lucrul cu dropbox.

Mai intii de toare ne asiguram ca avem instalat pachetul curl, daca nu, instalam:
(RedHat based OS)
# yum install curl

sau
(Debian Based OS)
# apt-get install curl

Copiem scriptul si dam drepturi de executie

$ wget https://raw.github.com/andreafabrizi/Dropbox-Uploader/master/dropbox_uploader.sh
$ mv ./dropbox_uploader.sh /usr/local/bin/dropbox_uploader.sh
$ chmod +x
/usr/local/bin/dropbox_uploader.sh

Ne logam in contul Dropbox si mergem in sectiunea Development unde cream o aplicatie:

1.drb0

2.drb23. Dupa crearea cu succes, aplicatia dropbox ne va returna 2 Application Key, de ele vom avea nevoie pentru a seta clientul pe serverul Linux.

drb1

4. Acum e momentul sa pornim scriptul copiat anterior:
$ ./dropbox_uploader.sh

ca rezultat, ca prima data rulat vom vedea un output asemanator care va cere sa introducem APP KEY si SECRET KEY, (care au fost generate anterior de pagina dropbox):
drb4

5. Dupa ce am raspuns la toate “intrebarile” scriptului, primim un URL de autentificare pe dropbox.

drb5

6. La acest pas terminam cu configurarea Client (Linux) – Dropbox.

Pentru verificarea conexiunii cu Dropbox executam:

$ ./dropbox_uploader.sh info
Ca output vom vedea informatia despre starea contului:

 

Pentru a vedea lista de comenzi executam scriptul fara nici o optiune.
$ ./dropbox_uploader.sh


Trecem la crearea scriptului de backup si upload:

Eu am creat 2 scripturi separate:
1. Backup a bazelor de date din MySQL
2. Backup a directoriilor care contin pagini web din /var/www/html/ (in acest directoriu, avem site1 site2 site3 etc.)

Script1.sh

 

Script2.sh

 

 

Scripturile sint create, acum avem nevoie sa le programam sa se execute la o anumita ora intr-o anumita zi.
Spre exemplu noi nu avem nevoie sa facem backup in fiecare zi la mapele cu website-uri, asa cum ele pot fi modificate rar, sau foarte rar.
In schimb baza de date se populeaza cu informatie noua foarte des, deci bazele de date sint cu mult mai critice in cazul unui hack, sau pierdere de informatie. DIn aceasta cauza setam backup-l pentru bazele de date in fiecare noapte spre ex. la ora 3 a.m. (Atunci cand serverul este sub load minim), Iar pentru paginile web o data pe saptamina asa cum continutul se schimba mai rar.

Pentru a pune ceva in scheduler – cron executa:

# crontab -e
in care adaugam timpul dorit de executare urmat de comanda care se va executa

In cazul nostru lista din crontab va arata in felul urmator:

#Pentru DB backup in fiecare zi la ora 3 a.m.
* 3 * * *  /bin/bash /scripts/script1.sh

#Pentru Web Site Backup, in fiecare simbata la ora 2 a.m.
* 2 * * */5  /bin/bash /scripts/script2.sh

Unde:
*/5 – in asa mod se indica executarea repetata. In cazul dat – in fiecare simbata.

 

[How to] Reset root password

Linux_GRUBAstazi voi descrie cateva metode de resetare a parolei pentru super userul ROOT.

Acest proces paote fi facut in 2 moduri:

1. Prin intermediul GRUB
2. Cu ajutorul unui LiveCD Linux

Trecem la practica pas cu pas:

Metoda 1: Cu ajutorul GRUB

1. Pornim/restartam PC-ul ca sa ajungem in meniul GRUB

Linux_GRUB

2. In cazul in care avem mai multe sisteme de operare – selectam linia care coincide cu SO la care dorim sa resetam parola.
3. Tastam 'e', primim access la editarea configuratiei de boot-are asemanator ca in imaginea de mai jos


Linux_GRUB_bash
4. In linia de configurare de boot-are a sistemului de operare gasim linia in care se alfa imaginea KERNEL-ui Linux, (in cazul meu e  Linux /boot/vmlinuz-3.2.0-4-486 ) adaugam la sfirsit stringul init=/bin/bash (ceia ce deja este ilustrat in imagine) dupa care tastam F11 sau CTRL+X pentru a incarca sistemul de operare cu modificare dorita.
Aceasta modificare este aplicata doar o singura data, deoarece noi am modificat interactiv configuratia de boot-are a sistemului. La urmatorul restart GRUB va incarca fisierul de configuratie de pe discul rigid.

5. In scurt timp vom vedea pe ecran un asemenea promt:linux_boot
6. Jumatate de pas este facut, acum este necesar de montat partitia / in regim ReadWrite (RW) si sa apelam comanda passwd pentru a modifica parola pentru root:

mount -o,remount rw / 

unde -o,remount rw / – optiunea de remontare a sistemului de fisiere in regim read-write

Dupa remontarea cu succes introducem urmatoarea comanda pentru a seta o parola noua pentru userul root:
passwd root
si introducem noua parola

Linux_ch_pass7. Ca urmare vedem "Password updated successfully" – ceia ce ne dovedeste ca parola a fost modificata cu succese.
8. Facem reboot si ne logam cu noua parola setata.

 

Metoda 2: LiveCD

Daca din oarecare motive nu ai acces la grub, sau fie ca grub e cu parola oricum nu e problema de resetat parola la root.

Pentru aceasta avem nevoie de un LiveCD cu orice distributiv linux, Debian,Ubuntu,Gentoo,CentOS etc.
1. Incarcam sistemul de operare RAM
2. Deschidem consola si montam partitia pe care e instalat SO Linux.
spre exemplu data partitia pe care e instalat Linux e sda2 executam urmatoarea comanda pentru a monta partita in mapa /media/
mount /dev/sda2 /media/

3. Facem chroot in partitia montata anterior.
(!) Comanda chroot pe un sistem de tip UNIX schimb─â în mod aparent directorul root pentru procesul curent ╚Öi pentru toate procesele derivate din acesta. Procesul nu poate accesa fi╚Öiere în afara directorului root asignat. Uneori, acest director setat cu chroot mai poart─â ╚Öi numele de chroot jail

chroot /media/ /bin/bash

5. Modificam parola utilizatorului cu comanada passwd si ca argument numele utilizatorului:
passwd root

Ca urmare vedem "Password updated successfully"

 

Enjoy.

Vi/Vim

vimAm întilnit foart─â mult─â lume, ╚Öi anume chiar utilizator de GNU/Linux care încearca s─â evite lucrul cu editorul VI sau VIM. La întrebarea ”De ce ?” primesc mereu acela╚Öi r─âspuns cu cuvinte cenzurate în partea softuluiu dat.

De ce oare to╚Ťi care il încerc─â sa-l foloseasc─â în linia de comanda (CLI) ╚Öi nu se l─âmuresc în primele 2 minute îl disconsider─â un produs util ? Raspunsul este simplu: VI/VIM nu este un editor intuitiv, ci unul modal cu o logica de func╚Ťionare mai deosebit─â. Prin urmare far─â cuno╚Ötin╚Ťe de baz─â e imposibil sa operezi cu acest editor la nivel de intui╚Ťie. Din aceast─â cauz─â mu╚Ťi care îl incearc─â în grab─â, sau deschid editorul ╚Öi mai apoi ne ╚Ötiind cum sa-l închid─â îl despre╚Ťuiesc. Ca urmare nu-l mai folosesc niciodat─â ╚Öi prefer─â o alternativ─â de genul nano, emacs etc.

Recunosc c─â prima impresie în privin╚Ťa lui VI la fel a fost negativ─â. Prima dat─â cînd am început sa studiez linux, ╚Öi cînd din gre╚Öeal─â am deschis vim, m-am intervat ca nu puteam ie╚Öi din el. Unica solu╚Ťie era butonul Reset, deoarece eram doar în consol─â, fara GUI. Mai tîrziu, am studiat editorul VIM mai detaliat, ╚Öi pot sa spun ca am r─âmas impresionat de posibilit─â╚Ťile sale sub linia de comand─â.

Deci de ce VI și VIM ?
VI – este un editor de text open source multiplatform─â.
VIM – Vine de la Vi IMproved, deriveaza de la editorul VI, dar cu un set larg de îmbun─ât─â╚Ťiri.

Ce inseamn─â editor modal ? – prin editor modal în╚Ťelegem c─â editorul are mai multe regimuri de func╚Ťionare. Cu alte cuvinte, acea╚Öi tast─â poate avea func╚Ťionalitate diferit─â în regimuri diferite. VI/VIM opereaz─â în 2 regimuri sau moduri de baz─â Normal Mode (Regim normal, sau regim de comand─â) ╚Öi Insert Mode (Regim de inserare).

1. Regim normal (Normal mode):  În acest regim fi╚Öierul deschis poate fi listat ╚Öi asupra textului se pot executa anumite modific─âri. Modific─ârile se fac cu ajutorul tastelor care ╚Öi sînt interpretatori de comenzi. Fiecare tast─â execut─â o anumit─â comand─â. Odat─â lansat editorul se afl─â în regimul normal sau în regimul de comand─â.

vim1 

2. Regim de inserare (Insert mode): Pentru a intra în regimul de inserare a textului, se tasteaz─â tasta i sau INSERT. În acest regim editorul permite inserarea textului, deci toate tastele sînt folosite pentru a culege text. Pentru a ie╚Öi din regimul de inserare în regimul de comanda (sau regim normal) tast─âm ESC sau CTRL+C

vim2

 

Cum aflu în care regim sînt la moment ?
Atragem aten╚Ťia în stînga jos al paginii, daca ne afl─âm în regimul de comand─â nu o s─â vedem nici o inscrip╚Ťie, îns─â dac─â ne vom afla în regimul de inserare vom vedea scris:
vim2


Mai jos vreau s─â descriu cele mai utilizate comenzi de care o sa ave╚Ťi nevoie:
(!) Toate aceste comenzi sunt valide în regimul normal (regimul de comand─â)


Comutarea intre modurile de lucru


Operatii asupra fisierelor


Pozitionarea cursorului


Comenzi folosind marcaje

Stergere

Toate comenzile de stergere copiaza textul sters in tampon de unde el se poate extrage cu comanda

p

 


Lucrul cu tamponul (Bufer)


Cautare si schimbare (Search & Replace)


Setarea diferitelor regimuri

 

Pentru lista completa a comenzilor ╚Öi posibilit─â╚Ťilor vim

Si ca bonus, recomand un cheatsheet sa-l puneti ca wallpaper ­čÖé

vim_cheatsheet

 

GNU/Linux + GSM Modem

3132009030311c86352Fiecare stie ce este un GSM modem, fie el de la orange, moldcell, sau de la alt operator de telefonie mobila. De regula o data conectat in portul USB a unui PC cu SO Windows, automat se starteaza procedura de instalare a aplicatie si a driverelor necesare de pe memoria interna a modemului, fie de pe CD-ul cu care vin in complectatie. Ce tine de SO GNU/Linux deseori intiminam probleme cu instalarea si configurarea a acestor dispozitive.

Da-ti sa ne descurcam ce si cum lucreaza:

De obicei asa tipuri de modemuri au 2 stari,
Regim┬á 1 – ca storage (fie memoria interna de pe care se instaleaza driverele, sau cea externa – flash)
Regim 2 – de modem

Pe windows:
Implicit, o data conectat, modemul lucreaza in Regimul 1 – ca flash. Ca urmare aceasta permite sa instalam driverele si aplicatia de conexiune cu internetul (PPP). Aplicatia o data pornita, comuteaza modemul in regimul 2 – regimul modem.

Pe linux:
Ce tine de Linux, aici configurarea si instalarea difera, dar logica de functionare e aceasi.
Pentru schimbul de regimuri in linux foloseste usb-modeswitch.
Deci daca el deja nu a fost instalat il instalam

Dupa instalare verificam si modificam dupa necesitate fisierul  de configurare /etc/usb_modeswitch.conf

Unde  DefaultVendor si  DefaultProduct trebuie sa coincida cu id-urile modemului vostru. Aceasta poate fi verificata cu ajutorul utilitei lsusb

Odata setat corect modemul este gata de functionare.

Spre exemplu cel mai simplu test de verificare a functionalitatii corecte a modemului este sa verificam starea contului (spre ex. *133#). Insa nu toate modemele pot trimite (encrtipta) corect numarul de apel mai ales pentru numere care se incepe cu caractere de genul * sau # pentru a trimite cereri USSD. Pentru asta este nevoie de codificat numarul dat.

Pentru aceasta am creat un script are converteste numarul *133#

Deci numarul *133# codificat este – AAD86C3602

Ca urmare trimitem o cerere USSD de stare a contului:

si pentru a decodifica informatia folosim scriptul de mai jos inlocuind cu stringul primit cuvintul  STRINGUL_AICI :

Scriptul final in bash este :

Internet 2G/3G/4G pe linux

Cum ramine cu internetul ? – Foarte simplu!Odata instalat corect modemul, avem nevoie doar de aplicatia care face conexiune cu operatorul, similara ca si pe windows. In linux este folosit packetul PPP

Dupa instalare creem fisierul de configurare (spre ex /etc/ppp/peers/my_isp)

 

Pentru informatie mai detaliata sau modeme specifice gautati pe google ­čÖé

Cum sa setezi Java SE pe Debian

Un mic howto sa instalez Java 7 (Oracle) pe Debian:

1. Copiem pachetul Java JDK de pe oracle.comwget
wget http://download.oracle.com/otn-pub/java/jdk/7/jdk-7-linux-x64.tar.gz

2. Dezarhivam in /usr/lib64/jvm/
tar zxvf jdk-7-linux-x64.tar.gz -C /usr/lib64/jvm/ 

Debian are o comanda utila pentru mentenanta a diferitor versiuni al asa packete ca java update-altenatives

3. Setam Java

update-alternatives –install /usr/bin/java java /usr/lib/jvm/jdk1.7.0/bin/java 1070
update-alternatives –install /usr/bin/javac javac /usr/lib/jvm/jdk1.7.0/bin/javac 1070
unde 1070 este prioritatea

pentru a verifica versiunea activa folosim:
update-alternatives –config java

ca rezultat primim: Selection Path Priority Status
————————————————————
* 0 /usr/lib/jvm/jdk1.7.0/bin/java 1070 auto mode
1 /usr/lib/jvm/java-6-openjdk/jre/bin/java 1069 manual mode
2 /usr/lib/jvm/jdk1.7.0/bin/java 1070 manual mode
Asa cum 1070 este mai mare, se va folosi versiunea 7.0, dar in caz de ceva tot asa se poate de modificat.

La final pentru a vedea versiunea Java care este activa in sistem
java -version


Profit!

Cum scriem imagini ISO pe CD/DVD din Consol─â

tux-and-cdLinux┬á –┬á Puterea liniei de comanda (CLI)

Ast─âzi voi scrie cum putem s─â scriem o imagine ISO pe CD/DVD din linia de comand─â:

Să purcedem pe pași:

1. Verfic─âm dac─â avem CD/DVD Writer

Ca output primim ceva asemanator

Dupa cum vedem, pe bus 1,0,0 avem connectat HDD-ul, pe bus 1,1,0 avem conectat un DVD-RAM. Ca urmare stiind adresa SCSI Bus pe care se afla dispozitivul DVD-RW trecem la pasul urmator.

2. Scriem ISO-ul pe CD/DVD


Unde:
-v 
= verbose
-eject  = după finisare CD-ul este scos
speed=8  = viteza scrierii
dev = adresa bus al DVD-RW
path = /locatie/imagine.iso (locatia imaginii ISO)