EnglishSvenska

ODE solver in Python using custom Forward-Euler

Predator-Pray

For realtime simulation it is important to have an ODE solver that is simple enough. I did not find any plug-and-play kind of ODE solvers for realtime applications so I wrote one myself. The greatest benefit of this approach is that you have full control of every time step. Good for realtime simulations.

https://github.com/sebnil/Python-ODE-Forward-Euler/blob/master/Forward-Euler%20ODE%20solver.ipynb

It is very simple but show the basics of how to write an ODE solver. It could be extended to use more sophisticated solvers like Runge-Kutta or Crank-Nicolson.

Tagged with:
Posted in Blog

Trying some game development in Python

I wanted to learn some basic game development in OpenGL and Python. Pyglet is very good for this kind of thing. Unfortunately it seems to have a less active community compared to Pygame (but in my opinion Pyglet seems much more Pythonic in its design). My game will not win any awards since it is a Asteroids clone. But it has some other features like:

  • Bullets have recoil and will slightly accelerate the ship rearwards when shooting.
  • Game graphics from a professional artist. (nalishac.com)
  • When I started this list I thought there would be more features but I think this is it.

pyglet-asteroids on Github

 

Using a lot of code and inspiration from:

Code structure and inspiration from:

Tagged with:
Posted in Blog

Infrared experiments to control the television from a website

Where is the remote? Wouldn't it be good if you could switch channels on the television from a website. A wanted to see if it is possible. It didn't.. Note to self; buy more powerful IR transmitters.

Posted in Arduino, Blog

Calculate Personal Financial worth with Python

I wanted to know how much my financial assets are worth across one or more bank accounts, so I created a small python script for it.

I just type all my assets into a list like this:

assets = {
'AAPL': {
'own': 40,
'current_value': yahoo_share('AAPL')
},
'Coca-Cola': {
'own': 35,
'current_value': yahoo_share('KO')
},
'NOF': {
'own': 170,
'current_value': yahoo_share('NOF.OL')
},
'Nordnet Superfonden Sverige': {
'own': 6.4099,
'current_value': morningstar("http://www.morningstar.se/Funds/Quicktake/Overview.aspx?perfid=0P0000J24W")
},
'SEB Japanfond': {
'own': 491.8263,
'current_value': morningstar("http://www.morningstar.se/Funds/Quicktake/Overview.aspx?perfid=0P00000LU7")
},    
}

Then I run the script and get an output looking a bit like this:
Skärmavbild 2015-09-12 kl. 13.40.51

(Not my actual positions 🙂 )

Next step is to program a get rich quick algorithm. Until then the code for calculating assets is published here:
https://github.com/sebnil/Pengar

Tagged with:
Posted in Blog

Responsive one-page WordPress site

I have not been doing wordpress sites for a while but today I changed that. Nalisha Chouraria needed a simple personal webpage and I got it done in a day by using some shortcuts (mainly Twitter bootstrap css).

nalishac.com
Skärmavbild 2015-09-06 kl. 18.50.15

 

Tagged with: ,
Posted in Blog

HIL testing in Arduino

I am trying out Hardware-in-the loop simulation on a new Arduino project for a client. Test driven development is unfortunately not very big in the Arduino community (yet) so I decided to implement something by myself. The setup is simple:

  1. One Arduino is running the application software.
  2. Another Arduino connects to inputs and outputs on the first Arduino. This Arduino will include all the test code.

2015-05-14 16.08.09

A test case could for example be, when the user presses a button a LED should light up. The second Arduino will output a signal to the button input on the first Arduino, and then check if the LED pin output is high. Example code is shown below.

void loop() {
// Simulate that user presses button
digitalWrite(BUTTON_PIN, 1);
// check that the LED lights up
assert(LED_PIN, 1);
delay(500)
// check that some actuator starts running
assert(ACTUATOR_PIN, 1);
// Simulate that user releases button
digitalWrite(BUTTON_PIN, 0);
// Led and actuator should turn off
assert(LED_PIN, 1);
assert(ACTUATOR_PIN, 1);
// stop execution
while (1) {}
}
bool assert(uint8_t pin, bool expectedState) {
bool state = digitalRead(pin);
if (state != expectedState) {
Serial.print("## FAILURE: pin ");
Serial.print(pin);
Serial.print(" == ");
Serial.print(state);
Serial.print(" != ");
Serial.print(expectedState);
Serial.println();
return false;
}
else {
Serial.print("## OK: pin ");
Serial.print(pin);
Serial.print(" == ");
Serial.print(state);
Serial.println();
return true;
}
}

Why the hassle?

It might seem unnecessary, (and it is for simple problems), but it does increase code quality and decreases the risk of bugs being introduced when writing new features to the code.

Making it better

I would like to write my test code in Python on a laptop and control an Arduino (via Firmata for example). Then I would have proper tools for testing and generating test reports. For now the Arduino solution is sufficient though.

Tagged with:
Posted in Arduino, Blog

Python for sftp and mysql backup

I needed to backup some sftp sites and mysql from a remote server to my local server at home. Piece of cake in Python.

After that I add it as a Jenkins script to schedule periodic backups:

Skärmavbild 2015-05-05 kl. 23.18.45 jenkins_no_bg

import shutil
import os
import paramiko
import pysftp
import select
import logging
logging.basicConfig(level=logging.DEBUG)
def sftp_backup(ssh_host=None, ssh_username=None, ssh_password=None, source_directory=None, local_directory=None):
with pysftp.Connection(ssh_host, username=ssh_username, password=ssh_password, log=True) as sftp:
sftp.chdir(source_directory)
# first remove the local directory to make room
try:
logging.info('Removing local directory: {}'.format(local_directory))
shutil.rmtree(local_directory)
logging.info('Done removing local directory: {}'.format(local_directory))
except:
logging.info('Can\'t delete {}. Probably does not exist'.format(local_directory))
# then create the directory
if not os.path.exists(local_directory):
logging.info('Creating empty local directory: {}'.format(local_directory))
os.makedirs(local_directory)
logging.info('Done creating local directory: {}'.format(local_directory))
# recursively copy to local_directory
logging.info('Starging to download from {} to {}'.format(source_directory, local_directory))
sftp.get_r('', local_directory)
logging.info('Done')
def dump_mysql_to_file_via_ssh(ssh_host=None, ssh_user=None, ssh_password=None, mysql_host=None, mysql_user=None,
mysql_password=None, mysql_databases=None, output_file='dump.sql'):
logging.debug('dump_mysql_to_file_via_ssh')
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(hostname=ssh_host, username=ssh_user, password=ssh_password)
stdin, stdout, stderr = ssh.exec_command('mysqldump --host={mysql_host} -u {mysql_user} -p{mysql_password} --databases {mysql_databases}'.format(
mysql_host=mysql_host,
mysql_user=mysql_user,
mysql_password=mysql_password,
mysql_databases=mysql_databases
))
logging.info('Begin writing to file {}'.format(output_file))
file = open(output_file, 'w')
# Wait for the command to terminate
while not stdout.channel.exit_status_ready():
# Only print data if there is data to read in the channel
if stdout.channel.recv_ready():
rl, wl, xl = select.select([stdout.channel], [], [], 0.0)
if len(rl) > 0:
# Print data from stdout
r = stdout.channel.recv(1024)
file.write(str(r))
file.close()
logging.info('Done writing to file.')
ssh.close()
if __name__ == '__main__':
dump_mysql_to_file_via_ssh(
ssh_host='ssh.example.com',
ssh_user='',
ssh_password='',
mysql_host='',
mysql_user='',
mysql_password='',
mysql_databases='',
output_file='sebastiannilsson.com.sql'
)
sftp_backup(ssh_host='ssh.example.com', ssh_username='', ssh_password='', source_directory='', local_directory='')
Tagged with:
Posted in Blog

Cpac Systems: Programming for outboard steering and propulsion systems

The project involved solving complex problems and also lots of boat testing around the world (Sweden, Japan and the US). My main tasks were to implement software algorithms, and as a side project I also developed a boat simulator to reduce time spent doing boat testing.

Posted in Timeline

Algorithmic trading with Python

I have been experimenting with algorithmic trading for a couple of weeks. Zipline is a Python library for backtesting trading algorithms and I would like to share one of the algorithms I made.

import talib
from zipline.api import record, order_target, history, add_history
import dateutil
import logging
from zipline.utils.factory import load_from_yahoo
from zipline.finance.slippage import FixedSlippage
from zipline.algorithm import TradingAlgorithm
from zipline.finance import commission
logging.basicConfig(level=logging.DEBUG)
# initialize algorithm
def initialize(context):
logging.debug('enter initialize')
context.set_slippage(FixedSlippage())
context.set_commission(commission.PerTrade(cost=5))
context.LOW_RSI = initialize.low_RSI
context.HIGH_RSI = initialize.high_RSI
context.rsi_window = initialize.rsi_window
add_history(context.rsi_window, '1d', 'price')
context.i = 0
context.invested = False
# default parameters for algorithm
initialize.rsi_window = 15
initialize.low_RSI = 30
initialize.high_RSI = 70
# Will be called on every trade event for the securities you specify.
def handle_data(context, data):
logging.debug('enter handle_data')
context.i += 1
if context.i < context.rsi_window:
return
# get the last RSI value
prices = history(context.rsi_window, '1d', 'price')
sec_rsi = talib.RSI(
prices[context.security].values,
timeperiod=context.rsi_window - 1)
# buy and sell flags
buy = False
sell = False
if sec_rsi[-1] < context.LOW_RSI and not context.invested:
# RSI under 30 indicates oversold, time to buy
order_target(context.security, 1000)
logging.debug('Buying {}'.format(context.security))
context.invested = True
buy = True
elif sec_rsi[-1] > context.HIGH_RSI and context.invested:
# RSI over 70 indicates overbought, sell everything
order_target(context.security, 0)
logging.debug('Selling {}'.format(context.security))
context.invested = False
sell = True
# record data for each time increment
record(secRSI=sec_rsi[-1],
price=data[context.security].price,
buy=buy,
sell=sell)
logging.info(context.portfolio.cash)
def run_algorithm(
security='AAPL',
start_date='20100101',
end_date='20150101',
initial_cash=100000,
rsi_window=15,
low_RSI=30,
high_RSI=70):
logging.debug('run_algorithm begin')
# dates
start = dateutil.parser.parse(start_date)
end = dateutil.parser.parse(end_date)
# get data from yahoo
data = load_from_yahoo(stocks=[security], indexes={}, start=start, end=end)
logging.debug('done loading from yahoo. {} {} {}'.format(
security, start_date, end_date))
# create and run algorithm
algo = TradingAlgorithm(
initialize=initialize,
handle_data=handle_data,
capital_base=initial_cash)
algo.security = security
initialize.low_RSI = low_RSI
initialize.high_RSI = high_RSI
initialize.rsi_window = rsi_window
logging.debug('starting to run algo...')
results = algo.run(data).dropna()
logging.debug('done running algo')
return results
if __name__ == '__main__':
import matplotlib.pyplot as plt
# run algorithm and get results
results = run_algorithm(
security='AAPL',
start_date='20100101',
end_date='20150101',
initial_cash=100000,
rsi_window=15,
low_RSI=30,
high_RSI=70)
# get s&p500 and nasdaq indexes
index_data = load_from_yahoo(
stocks=['^gspc', '^ixic'],
indexes={},
start=results.index[0],
end=results.index[-1])
# portfolio value, stock holdings and S&P 500 index
fig = plt.figure(figsize=(12, 6))
ax11 = fig.add_subplot(311)
ax12, ax13 = ax11.twinx(), ax11.twinx()
ax13.spines['right'].set_position(('axes', 1.07))
ax11.set_ylabel('portfolio value', color='blue')
ax12.set_ylabel('holdings', color='green')
ax13.set_ylabel('S&P 500', color='red')
# portfolio value
ax11.plot(results.index, results.portfolio_value, color='blue')
# holdings (number of stocks owned)
holdings = [0 if t == [] else t[0]['amount'] for t in results.positions]
ax12.plot(results.index, holdings, color='green')
ax12.set_ylim([min(holdings) - 30, max(holdings) + 30])
# index
ax13.plot(index_data.index, index_data['^gspc'], color='red')
# algo visualization
ax21 = fig.add_subplot(312)
ax21.set_ylabel('stock price', color='blue')
ax22 = ax21.twinx()
ax22.set_ylabel('rsi', color='red')
# stock
ax21.plot(results.index, results.price, color='blue')
# add sell and buy flags on top of stock price
ax21.plot(
results.ix[results.buy].index,
results.price[results.buy],
'^',
markersize=10,
color='green')
ax21.plot(
results.ix[results.sell].index,
results.price[results.sell],
'v',
markersize=10,
color='red')
# rsi value
ax22.plot(results.index, results.secRSI, color='red')
# add lines to show under- and over value indicator
ax22.plot([results.index[0], results.index[-1]], [30, 30], 'k-')
ax22.plot([results.index[0], results.index[-1]], [70, 70], 'k-')
# portfolio value, stock value and index in percentage
ax31 = fig.add_subplot(313)
ax32, ax33 = ax31.twinx(), ax31.twinx()  # share x for other plots
ax31.set_ylabel('algo %', color='blue')
ax32.set_ylabel('snp index %', color='green')
ax33.set_ylabel('stock %', color='red')
ax33.spines['right'].set_position(('axes', 1.07))
# portfolio value
ax31.plot(
results.index,
results.portfolio_value / results.portfolio_value[0] * 100 - 100,
color='blue')
# index
ax32.plot(
index_data.index,
index_data['^gspc'] / index_data['^gspc'][0] * 100 - 100,
color='green')
# stock value
ax33.plot(
results.index,
results.price /
results.price[0] * 100 - 100,
color='red')
plt.show()

If you get it working you should see a plot similar to this one:
Skärmavbild 2015-04-04 kl. 22.20.20

If you are observant, it is easy to see that the performance of this algorithm is not good enough to be used on a real portfolio, and it is more of a test.

Links:

Tagged with: ,
Posted in Blog

Wall computer

What to do with a worn out Samsung series 9 ultraslim laptop? Turn it into a wall computer! Perfect for showing weather and traffic information.

Tagged with: , ,
Posted in Blog