Implement a GeneralOptimizer class for new topologies

[whzup]

As proposed in #147 we could implement a new GeneralOptimizer class in addition to the two existing GlobalBest and LocalBest classes. In this class, it will then be possible to select a custom topology for the optimization. We could also include a foundation for multilayer PSO (as proposed in #75).

The only issue I see is that a GeneralOptimizer might make the GlobalBest or LocalBest classes obsolete as it might optimize faster and/or run with less memory consumption especially if we implement higher-level topologies such as Von Neumann or Pyramid (#129).

[ljvmiranda921]

A GeneralOptimizer is good, just plug-in your desired topology, and optimize away. But I'm on-the-fence that it'll make GlobalBest or LocalBest obsolete, we can keep these optimizers going forward. I see the GeneralOptimizer as a higher-order abstraction of GlobalBest / LocalBest, so this may be good. Any thoughts @SioKCronin ?

We could also include a foundation for multilayer PSO (as proposed in #75)

Is multilayer PSO "just a different topology?" I read the paper and a part of me thinks it is, but I might be wrong. Still not sure though, might require a second or third opinion (@SioKCronin , maybe @CPapadim can also weigh-in?).

For now I think let's go implementing GlobalOptimizer under the single module, have it inherit from SwarmOptimizer, and have a topology as its attribute.

import pyswarms as ps
from pyswarms.backend.topology import Foo

my_topology = Foo()
optimizer = ps.single.GlobalOptimizer(topology=my_topology, options)

[ljvmiranda921]

I'd love a documentation for this where we just animate swarm trajectories using different topologies as they traverse through a fairly difficult objective function 💯

[whzup]

I might start working on this on the weekend after I finished my exams if you like. I have holidays after that so I'd be up for some work on this project 😄.
I like the idea but I'm not sure about the viability of the GlobalBest and LocalBest classes after implementing it so I'd love to hear some other opinions! But in the end, you're the boss @ljvmiranda921 so I'd do it in whatever way you wish 😋.

[ljvmiranda921]

Awesome! Looking forward to it and I wish you well in your exams!

[whzup]

Hey @ljvmiranda921, I have created a GeneralOptimizer class with a topology parameter. At the moment it's actually more or less a copy of the GlobalBest class with custom topology choice. I think we can use this class for future features that we don't want to include in either the GlobalBest or LocalBest class. I'm going to add an exception for the topology parameter in case one does not input the right type and I'll try to create an animation for the 3 existing topologies. And here some questions:

• There is already a feature to plot the particle animations right?
• How did you make your GIF in the Inverse Kinematics tutorial on your blog?
• Should I include a special case for when someone chooses a Ring topology? (with the additional values)
• Is there something else we can already include at the moment?

As far as multilayer PSO goes I have left a comment in #75 where I proposed an idea for the implementation (if we even want to include it at the moment)

NOTE: The error in the Pyramid class still exists and should not go unnoted! I made the Pyramid topology the default topology so we won't forget it hopefully.

[ljvmiranda921]

Hi @whzup , thanks for stepping up, I really appreciate your enthusiasm. To answer your questions first:

1. Yup, use the development branch's plotters module. It can plot 2-D swarms via contour and surface plots.
2. I did this one manually, you can check the Jupyter notebook example in visualization (development branch) and I called some plt.savefig functions to render a GIF animation.
3. You mean the Ring-plus-static topology? As of now, it's not implemented yet, let's just make do with the dynamic topology.
4. For now, we only have the Ring, Star, and the Pyramic topology. No need to include multilayer for now, in my opinion, it may take some time and perhaps we can focus on GeneralOptimizer first.

Noted on the Pyramid class, just a few questions:

1. What is the error again? Can you paste the traceback here?
2. Is this a regular bug that renders it unusable or an edge-case?

Again, thanks a lot! We really appreciate this!

[whzup]

Regarding your answer No.3: No, I actually meant the dynamic topology with the two additional options for the distance and the number of neighbours, I'll include this one 😄.

Regarding the error:
If I use it as it is:

try:
            # If there are less than 5 particles they are all connected
            if swarm.n_particles < 5:
                best_pos = swarm.pbest_pos[np.argmin(swarm.pbest_cost)]
                best_cost = np.min(swarm.pbest_cost)
            else:
                pyramid = Delaunay(swarm.position)
                indices, index_pointer = pyramid.vertex_neighbor_vertices
                # Insert all the neighbors for each particle in the idx array
                idx = np.array([index_pointer[indices[i]:indices[i+1]] for i in range(swarm.n_particles)])
>               idx_min = swarm.pbest_cost[idx].argmin(axis=1)

E               IndexError: arrays used as indices must be of integer (or boolean) type

The error occurs because idx is an array of floats. But if I change the array data type to int:

try:
            # If there are less than 5 particles they are all connected
            if swarm.n_particles < 5:
                best_pos = swarm.pbest_pos[np.argmin(swarm.pbest_cost)]
                best_cost = np.min(swarm.pbest_cost)
            else:
                pyramid = Delaunay(swarm.position)
                indices, index_pointer = pyramid.vertex_neighbor_vertices
                # Insert all the neighbors for each particle in the idx array
>               idx = np.array([index_pointer[indices[i]:indices[i+1]] for i in range(swarm.n_particles)]).astype(int)

E               ValueError: setting an array element with a sequence.

This error occurs because the array is not filled with arrays of equal length (see this StackOverflow question for a reference. It has an answer but it's quite an ugly workaround). The whole function:

        try:
            # If there are less than 5 particles they are all connected
            if swarm.n_particles < 5:
                best_pos = swarm.pbest_pos[np.argmin(swarm.pbest_cost)]
                best_cost = np.min(swarm.pbest_cost)
            else:
                pyramid = Delaunay(swarm.position)
                indices, index_pointer = pyramid.vertex_neighbor_vertices
                # Insert all the neighbors for each particle in the idx array
                idx = np.array([index_pointer[indices[i]:indices[i+1]] for i in range(swarm.n_particles)]).astype(int)
                idx_min = swarm.pbest_cost[idx].argmin(axis=1)
                best_neighbor = idx[np.arange(len(idx)), idx_min]

                # Obtain best cost and position
                best_cost = np.min(swarm.pbest_cost[best_neighbor])
                best_pos = swarm.pbest_pos[
                    np.argmin(swarm.pbest_cost[best_neighbor])
                ]

For less than 5 particles it just behaves like the Star topology because all the particles are connected in that case and the Delaunay triangulation does not work with less than 5 points. So the error just occurs when there are more than 5 particles.

I've got another thing that just came to my mind. How should I include the animations in the documentation? In form of a Jupyter notebook?

[ljvmiranda921]

I’ll check the Pyramid one and see if we can fix this. Sure, I suggest (if you are fine) to play with the parameters (what if it’s highly social or highly cognitive?). The same with LocalBestPSO. For the documentation in read the docs, a GIF file is good. In the actual Jupyter notebook, just render it as it is similar to the one we’ve been doing in visualization.ipynb

On Sat, 30 Jun 2018 at 23:04 Aaron ***@***.***> wrote: Regarding your answer No.3: No, I actually meant the dynamic topology with the two additional options for the distance and the number of neighbours, I'll include this one 😄. Regarding the error: If I use it as it is: try: # If there are less than 5 particles they are all connected if swarm.n_particles < 5: best_pos = swarm.pbest_pos[np.argmin(swarm.pbest_cost)] best_cost = np.min(swarm.pbest_cost) else: pyramid = Delaunay(swarm.position) indices, index_pointer = pyramid.vertex_neighbor_vertices # Insert all the neighbors for each particle in the idx array idx = np.array([index_pointer[indices[i]:indices[i+1]] for i in range(swarm.n_particles)])> idx_min = swarm.pbest_cost[idx].argmin(axis=1) E IndexError: arrays used as indices must be of integer (or boolean) type The error occurs because idx is an array of floats. But if I change the array data type to *int*: try: # If there are less than 5 particles they are all connected if swarm.n_particles < 5: best_pos = swarm.pbest_pos[np.argmin(swarm.pbest_cost)] best_cost = np.min(swarm.pbest_cost) else: pyramid = Delaunay(swarm.position) indices, index_pointer = pyramid.vertex_neighbor_vertices # Insert all the neighbors for each particle in the idx array> idx = np.array([index_pointer[indices[i]:indices[i+1]] for i in range(swarm.n_particles)]).astype(int) E ValueError: setting an array element with a sequence. This error occurs because the array is not filled with arrays of equal length (see this StackOverflow question <https://stackoverflow.com/questions/43146266/convert-list-of-lists-with-different-lengths-to-a-numpy-array> for a reference. It has an answer but it's quite an ugly workaround). The whole function: try: # If there are less than 5 particles they are all connected if swarm.n_particles < 5: best_pos = swarm.pbest_pos[np.argmin(swarm.pbest_cost)] best_cost = np.min(swarm.pbest_cost) else: pyramid = Delaunay(swarm.position) indices, index_pointer = pyramid.vertex_neighbor_vertices # Insert all the neighbors for each particle in the idx array idx = np.array([index_pointer[indices[i]:indices[i+1]] for i in range(swarm.n_particles)]).astype(int) idx_min = swarm.pbest_cost[idx].argmin(axis=1) best_neighbor = idx[np.arange(len(idx)), idx_min] # Obtain best cost and position best_cost = np.min(swarm.pbest_cost[best_neighbor]) best_pos = swarm.pbest_pos[ np.argmin(swarm.pbest_cost[best_neighbor]) ] For less than 5 particles it just behaves like the Star topology because all the particles are connected in that case and the Delaunay triangulation does not work with less than 5 points. So the error just occurs when there are more than 5 particles. I've got another thing that just came to my mind. How should I include the animations in the documentation? In form of a Jupyter notebook? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#148 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AMWYs6XXyUoYrdDzJVdqSjr2RESKdyPrks5uB4VigaJpZM4U4Z8k> .

-- Lester James V. Miranda BS Electronics and Communications Engineering, Minor in Philosophy (Class of 2016) Ateneo de Manila University lester.miranda@ <[email protected]>*o <http://toki.waseda.jp>bf.ateneo.edu <http://bf.ateneo.edu>*

[whzup]

Ok, I resolved the problem with the pyramid topology😄. The idx array actually had arrays of type int32 in it! The problem was that the idx array itself had the data type object that was the first error I mentioned above. I worked around that by using a list comprehension. I did the same for the best_neighbor because the indexing there was actually for a 2D-array:

idx = np.array([index_pointer[indices[i]:indices[i+1]] for i in range(swarm.n_particles)])
idx_min = np.array([swarm.pbest_cost[idx[i]].argmin() for i in range(idx.size)])
best_neighbor = np.array([idx[i][idx_min[i]] for i in range(idx.size)]).astype(int)

All the tests for the optimizers pass now! I'm going to make a PR so you can review it.

By the way, I have an idea for the visualization. We could use this as another first-timers-only issue it might be a bit harder than the last one but I think it would be fitting to get the first insight into the project. What do you think? I really like that there are projects with the first-timers-only badge it made it way easier to find a project that I could contribute to when I searched for one.

[ljvmiranda921]

Wow, you're on a roll. Sure, open up a PR and let's discuss over there.
I agree with you on this. I will open up a first timers issue for this documentation. Let's finish the GeneralOptimizer first and after merging, we can do the latter. 👍

[whzup]

@ljvmiranda921 I have a question regarding the testing. There are these conftest.py files. For the optimizers, there are tests for the histories in it. Is there a way I can access the results of these tests, i.e. the history of every one optimization? They are fixtures so there must be a way to do this, right?

[ljvmiranda921]

Hi @whzup , which particular fixture are you pointing at? If it returns an Optimizer class, then we can simply obtain the values or results via the attribute? (Not sure if we're referring to the same thing though). If none of the current features work, maybe we can create new ones. What are you planning to test and how?