Neural Network Program

Share Tips, Code Samples, etc. with the Visual Prolog community.
Post Reply
User avatar
Thomas Linder Puls
VIP Member
Posts: 2327
Joined: 28 Feb 2000 0:01

Neural Network Program

Post by Thomas Linder Puls » 30 Jun 2008 7:30

Disclaimer: I do not know much about neural networks.

At VIP-ALC 2008 Paul Cherkez told about his experiences with neural networks. He has also made some mails in the Vip7 forum about it.

His presentation triggered me to find a solution to some performance problems he had. Using objects the net construction took very long time, but the calculation was rather fast. Using a more traditional fact approach the building was much faster, but calculation was slow. Both things as the nets grew larger.

The program is described in the wiki article Neural Network Program.
Attachments
neuralNet.zip
neuralNet project
(19.6 KiB) Downloaded 1929 times
Regards Thomas Linder Puls
PDC

Paul Cerkez
VIP Member
Posts: 380
Joined: 6 Mar 2000 0:01

Post by Paul Cerkez » 23 Feb 2012 13:03

the NN sample program Thomas provided was a good start and gave me a number of good ideas to get around some of the limitations I was experiencing.

while design of the NN architecture was an influcence in some networks, the actual implementaiton had more of an impact as hte networks became larger.

SOMs were the most processor intensive and computing increased exponentailly with increases in size
Back Props were fast and design had a huge impact on processing
Neural Abstraction Pyramid (NAP) - while design had some minor imapct, size and implementation were the driving factors.
Hybrid Custom Network (HCN) - Same as the NAP.

THe SOM, NAP and HCN were all capabile of unsupervised training. The final implementation of the HCN used semi-supervised training.

P.
AI Rules!
P.

lebronadamas
Posts: 1
Joined: 4 Apr 2012 6:32

Post by lebronadamas » 4 Apr 2012 6:44

I am looking for its complete architecture if you can share it would be great.

Paul Cerkez
VIP Member
Posts: 380
Joined: 6 Mar 2000 0:01

Post by Paul Cerkez » 4 Apr 2012 12:05

there are many different NN architectures, each one better suited for different problem domains.

what kind of problem are you trying to solve?
AI Rules!
P.

User avatar
Thomas Linder Puls
VIP Member
Posts: 2327
Joined: 28 Feb 2000 0:01

Post by Thomas Linder Puls » 13 Apr 2013 23:40

Updated for Visual Prolog 7.4
Attachments
neuralNet.zip
Updated for Visual Prolog 7.4
(19.49 KiB) Downloaded 621 times
Regards Thomas Linder Puls
PDC

whsheng
Posts: 1
Joined: 21 Oct 2013 3:24

Post by whsheng » 21 Oct 2013 3:30

Hi,

I tried this program, but it has unsuccessfully been compiled because of lack of files from "pfc\profile". What is the problem about? Thank you.

Whsheng

User avatar
Thomas Linder Puls
VIP Member
Posts: 2327
Joined: 28 Feb 2000 0:01

Post by Thomas Linder Puls » 21 Oct 2013 9:25

The profile package is used to measure the times spent in various parts of the program. But the profile package is only present in the Commercial Edition.

However, measuring execution times is of course not essential to the neural network problem itself. And can simply be removed from the program.

If you remove profiling from the program the run predicate should end looking like this:

Code: Select all

clauses     run():-         console::init(),         N1 = pyramidBuilder::new(layers):net,         N1:calculate(),         netSaver::new(N1):save(netFile1),         NL2 = netLoader::new(netFile1),         N2 = NL2:net,         N2:calculate(),         netSaver::new(N2):save(netFile2).
Besides this change you should remove all packages that the IDE says it cannot find and you should delete the include directory for the profile package that gives a file not found error.

Given these changes the program will run on the Personal Edition.

(I cannot recall what this program does except building a pyramid shaped neural network.)
Regards Thomas Linder Puls
PDC

Paul Cerkez
VIP Member
Posts: 380
Joined: 6 Mar 2000 0:01

Post by Paul Cerkez » 21 Oct 2013 11:56

Thomas,
The NN example provided a construct for a developer to use to build a pyramid based NNs.

It definitely solved the performance issues I was experiencing. I took your base example and modified/extended it to support the various pryamid architectures I was experimenting with.

For example, I built a NAP with an input layer of 256x256 (i x j). It had 7 'processing' layers with K feature arrays (comprised of i x j neural nodes per array) per layer. The layers decrease N/2. But in a NAP while the i x j dimension is decreasing, the number of feature arrays (K) in a layer increases by K*2 as you travel up the pyramid.

L(0) 256x256, K=0
L(1) 128x128, K =2
L(2) 64x64, K =4
etc.

For a 256x256 input layer, you can expect approx 469,000 neurons in the net (with all their weighted connections) and reducing to 128x128, the total drops to approx 117,000. (a factor of 4 change in total net size)

It had sufficiently adequate performance for my research purposes. Planning on experimenting with trying to implement a CUDA capable version in the near future to improve processing speed for a 'production-like environment' implementation.

P.
AI Rules!
P.

Post Reply