September 2017 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] What is the difference between shift and target in SLEPc ?
by Franck Houssen 25 Sep '17

25 Sep '17

OK, thanks, this is helpful. If I got you correctly: beforehand, there is no way to know exactly what the eigen values are. If it turns out that an eigen value makes A-sigma*I or A-sigma*B singular, then the solve may break. If so, afterwards, it's possible to change slightly the shift to avoid solve break down (but there is no way to know that beforehand). Franck ----- Mail original ----- > De: "Jose E. Roman" <jroman(a)dsic.upv.es> > À: "Franck Houssen" <franck.houssen(a)inria.fr> > Cc: "For users of the development version of PETSc" <petsc-dev(a)mcs.anl.gov> > Envoyé: Lundi 25 Septembre 2017 14:50:48 > Objet: Re: [petsc-dev] What is the difference between shift and target in SLEPc ? > > > > El 25 sept 2017, a las 13:21, Franck Houssen <franck.houssen(a)inria.fr> > > escribió: > > > > What is the difference between shift and target in SLEPc ? Shift > > (STSetShift) is clear to me, but, target (EPSSetTarget) is not. > > Can somebody give an example where one want/need to have a target which > > would be different from the shift ? > > > > Franck > > In shift-and-invert the shift is equal to the target by default. The target > is what you use to indicate where you want the eigenvalues to be sought (it > can be used without shift-and-invert). Normal usage is having both values > equal. If the target is exactly equal to an eigenvalue, then you may want to > perturb the shift (change it to a slightly different value) in order to > avoid a singular matrix A-sigma*I in the linear solves. (Some solvers such > as MUMPS do not have problems with singular matrices, so this is not > necessary in that case). > > Jose > >

1 0

Re: [petsc-dev] What is the difference between shift and target in SLEPc ?
by Jose E. Roman 25 Sep '17

25 Sep '17

> El 25 sept 2017, a las 13:21, Franck Houssen <franck.houssen(a)inria.fr> escribió: > > What is the difference between shift and target in SLEPc ? Shift (STSetShift) is clear to me, but, target (EPSSetTarget) is not. > Can somebody give an example where one want/need to have a target which would be different from the shift ? > > Franck In shift-and-invert the shift is equal to the target by default. The target is what you use to indicate where you want the eigenvalues to be sought (it can be used without shift-and-invert). Normal usage is having both values equal. If the target is exactly equal to an eigenvalue, then you may want to perturb the shift (change it to a slightly different value) in order to avoid a singular matrix A-sigma*I in the linear solves. (Some solvers such as MUMPS do not have problems with singular matrices, so this is not necessary in that case). Jose

1 0

What is the difference between shift and target in SLEPc ?
by Franck Houssen 25 Sep '17

25 Sep '17

What is the difference between shift and target in SLEPc ? Shift (STSetShift) is clear to me, but, target (EPSSetTarget) is not. Can somebody give an example where one want/need to have a target which would be different from the shift ? Franck

1 0

Re: [petsc-dev] (no subject)
by Jakub Kruzik 25 Sep '17

25 Sep '17

Thank you, Barry. We really appreciate it. Jakub On 23.9.2017 16:52, Barry Smith wrote: > Jakub, > > This is great, thanks for the information. I've added links from the PETSc main webpage to your work. > > Barry > >> On Sep 23, 2017, at 9:26 AM, Jakub Kruzik <jakub.kruzik(a)vsb.cz> wrote: >> >> Dear all, >> >> I would just like to note that we also develop SVM implementation. It is intended for large-scale datasets and makes use of PETSc parallel linear algebra. Currently, it supports only linear kernels - Hessian is, in fact, MATNORMAL with arbitrary underlying data matrix - it is, e.g. possible to use MATDENSE or MATAIJ depending on the problem. For the solution of the arising quadratic program (QP), it uses solvers from our PermonQP package. Both PermonSVM and PermonQP are libraries depending on PETSc. They are written in the PETSc coding style, pretty much like SLEPc. >> >> http://permon.it4i.cz/permonqp.htm >> http://permon.it4i.cz/permonsvm.htm >> >> https://github.com/it4innovations/permon >> https://github.com/it4innovations/permonsvm >> >> So far, PermonQP only implements an Augmented Lagrangian type algorithm which can be combined with any solver for box-constrained QP. In PermonQP, there are some concrete ones and also TAO wrapper. However, adding an Interior Point implementation is interesting for us as well. >> >> PermonSVM is so far a proof-of-concept thing, but it already scales pretty well (almost proportionally to the application of the data matrix to a vector). See, e.g. our PASC poster https://www.researchgate.net/publication/318317204_PERMON_PASC17_Poster >> >> We'll be grateful for any feedback on this. >> >> Jakub >> >> >> On 22.9.2017 06:06, Richard Tran Mills wrote: >>> Thanks for sharing this, Barry. I haven't had time to read their paper, but it looks worth a read. >>> >>> Hong, since many machine-learning or data-mining problems can be cast as linear algebra problems (several examples involving eigenproblems come to mind), I'm guessing that there must be several people using PETSc (with SLEPc, likely) in this this area, but I don't think I've come across any published examples. What have others seen? >>> >>> Most of the machine learning and data-mining papers I read seem employ sequential algorithms or, at most, algorithms targeted at on-node parallelism only. With available data sets getting as large and easily available as they are, I'm surprised that there isn't more focus on doing things with distributed parallelism. One of my cited papers is on a distributed parallel k-means implementation I worked on some years ago: we didn't do anything especially clever with it, but today it is still one of the *only* parallel clustering publications I've seen. >>> >>> I'd love to 1) hear about what other machine-learning or data-mining applications using PETSc that others have come across and 2) hear about applications in this area where people aren't using PETSc but it looks like they should! >>> >>> Cheers, >>> Richard >>> >>> On Thu, Sep 21, 2017 at 12:51 PM, Zhang, Hong <hongzhang(a)anl.gov> wrote: >>> Great news! According to their papers, MLSVM works only in serial. I am not sure what is stopping them using PETSc in parallel. >>> >>> Btw, are there any other cases that use PETSc for machine learning? >>> >>> Hong (Mr.) >>> >>>> On Sep 21, 2017, at 1:02 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: >>>> >>>> >>>> From: Ilya Safro isafro(a)g.clemson.edu >>>> Date: September 17, 2017 >>>> Subject: MLSVM 1.0, Multilevel Support Vector Machines >>>> >>>> We are pleased to announce the release of MLSVM 1.0, a library of fast >>>> multilevel algorithms for training nonlinear support vector machine >>>> models on large-scale datasets. The library is developed as an >>>> extension of PETSc to support, among other applications, the analysis >>>> of datasets in scientific computing. >>>> >>>> Highlights: >>>> - The best quality/performance trade-off is achieved with algebraic >>>> multigrid coarsening >>>> - Tested on academic, industrial, and healthcare datasets >>>> - Generates multiple models for each training >>>> - Effective on imbalanced datasets >>>> >>>> Download MLSVM at https://github.com/esadr/mlsvm >>>> >>>> Corresponding paper: Sadrfaridpour, Razzaghi and Safro "Engineering >>>> multilevel support vector machines", 2017, >>>> https://arxiv.org/pdf/1707.07657.pdf >>>> >>>

1 0

Re: [petsc-dev] PetscSF in Fortran
by Matthew Knepley 25 Sep '17

25 Sep '17

On Sun, Sep 24, 2017 at 8:57 PM, Adrian Croucher <a.croucher(a)auckland.ac.nz> wrote: > > > On 25/09/17 13:37, Adrian Croucher wrote: > >> >> Actually it looks like DMGetDefaultSF() does already work in Fortran >> though, and could be used to do the same thing. >> >> Do you think that would be a reasonable way to do it? >> >> > Hmm, I just tried it and it looks like the default SF produced by this > function doesn't have any overlap. So that won't work for me. > > Guess I might need a Fortran interface for PetscSFSetGraph/PetscSFGetGraph > after all. If you could put one it that would be great. > > Another simpler possibility that would work for my case is if there was a > DMCopySF() function for copying the point SF from one DM to another. Up to > you if you think that would be of enough general usefulness to warrant > inclusion. If you truly need the exact same SF for your grid, you should be able to use DMGet/SetPointSF() since it will just reference count it for you. Then the default SF is created automatically from the point SF. Does this work? Thanks, Matt > - Adrian > > -- > Dr Adrian Croucher > Senior Research Fellow > Department of Engineering Science > University of Auckland, New Zealand > email: a.croucher(a)auckland.ac.nz > tel: +64 (0)9 923 4611 > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.caam.rice.edu/~mk51/>

1 0

Re: [petsc-dev] PetscSF in Fortran
by Adrian Croucher 24 Sep '17

24 Sep '17

On 25/09/17 13:37, Adrian Croucher wrote: > > Actually it looks like DMGetDefaultSF() does already work in Fortran > though, and could be used to do the same thing. > > Do you think that would be a reasonable way to do it? > Hmm, I just tried it and it looks like the default SF produced by this function doesn't have any overlap. So that won't work for me. Guess I might need a Fortran interface for PetscSFSetGraph/PetscSFGetGraph after all. If you could put one it that would be great. Another simpler possibility that would work for my case is if there was a DMCopySF() function for copying the point SF from one DM to another. Up to you if you think that would be of enough general usefulness to warrant inclusion. - Adrian -- Dr Adrian Croucher Senior Research Fellow Department of Engineering Science University of Auckland, New Zealand email: a.croucher(a)auckland.ac.nz tel: +64 (0)9 923 4611

1 0

Re: [petsc-dev] PetscSF in Fortran
by Adrian Croucher 24 Sep '17

24 Sep '17

On 25/09/17 13:18, Adrian Croucher wrote: > > Alternatively, it occurred to me I could probably just use > DMCreateDefaultSF() to create the point SF on the new DM- would there > be any disadvantage in doing it that way? > > However, at the moment I can't, because it appears the Fortran > interface for DMCreateDefaultSF() is missing. I would have thought the > interface for that should be simple enough, as it just takes a DM and > a couple of PetscSections. Actually it looks like DMGetDefaultSF() does already work in Fortran though, and could be used to do the same thing. Do you think that would be a reasonable way to do it? - Adrian -- Dr Adrian Croucher Senior Research Fellow Department of Engineering Science University of Auckland, New Zealand email: a.croucher(a)auckland.ac.nz tel: +64 (0)9 923 4611

1 0

Re: [petsc-dev] PetscSF in Fortran
by Adrian Croucher 24 Sep '17

24 Sep '17

hi On 21/09/17 15:15, Jed Brown wrote: > Unfortunately PetscSFSetGraph/PetscSFGetGraph need custom bindings for > Fortran and they have not been written. Shouldn't be difficult, but > will take a little work/testing. I would probably make the array of > PetscSFNode be a PetscInt array of length 2n (or of shape (2,n)) -- > that's an equivalent memory representation to what we're using now. That sounds like the sort of thing I need. Alternatively, it occurred to me I could probably just use DMCreateDefaultSF() to create the point SF on the new DM- would there be any disadvantage in doing it that way? However, at the moment I can't, because it appears the Fortran interface for DMCreateDefaultSF() is missing. I would have thought the interface for that should be simple enough, as it just takes a DM and a couple of PetscSections. - Adrian > > Adrian Croucher <a.croucher(a)auckland.ac.nz> writes: > >> hi >> >> For the dual-porosity stuff I'm working on, I'm creating a modified >> DMPlex. Looking at e.g. TS ex11.c (in the SplitFaces() routine), it >> appears I am going to have to set up a PetscSF for the new DM (from what >> I can gather, that is needed for managing parallel communication of >> partition ghost cell values). >> >> The way I've set up the modified DMPlex, the partition ghost cells are >> in the same locations as they were in the original single-porosity DM. >> So I'm thinking I basically just have to copy the SF straight over from >> the original DM. >> >> The routine DMGetPointSF() seems to work OK in Fortran, but I'm having a >> bit of trouble with PetscSFGetGraph(). >> >> When I try to declare an array of type PetscSFNode for the iremote >> parameter, it doesn't seem to know about PetscSFNode, even though I have >> a 'use petsc' in my code, which normally gives access to everything. >> >> Also I'm not sure of exactly how to set up the ilocal and iremote array >> parameters for this function in Fortran. How should they be declared- as >> pointer arrays? >> >> - Adrian -- Dr Adrian Croucher Senior Research Fellow Department of Engineering Science University of Auckland, New Zealand email: a.croucher(a)auckland.ac.nz tel: +64 (0)9 923 4611

1 0

Re: [petsc-dev] (no subject)
by Barry Smith 23 Sep '17

23 Sep '17

Jakub, This is great, thanks for the information. I've added links from the PETSc main webpage to your work. Barry > On Sep 23, 2017, at 9:26 AM, Jakub Kruzik <jakub.kruzik(a)vsb.cz> wrote: > > Dear all, > > I would just like to note that we also develop SVM implementation. It is intended for large-scale datasets and makes use of PETSc parallel linear algebra. Currently, it supports only linear kernels - Hessian is, in fact, MATNORMAL with arbitrary underlying data matrix - it is, e.g. possible to use MATDENSE or MATAIJ depending on the problem. For the solution of the arising quadratic program (QP), it uses solvers from our PermonQP package. Both PermonSVM and PermonQP are libraries depending on PETSc. They are written in the PETSc coding style, pretty much like SLEPc. > > http://permon.it4i.cz/permonqp.htm > http://permon.it4i.cz/permonsvm.htm > > https://github.com/it4innovations/permon > https://github.com/it4innovations/permonsvm > > So far, PermonQP only implements an Augmented Lagrangian type algorithm which can be combined with any solver for box-constrained QP. In PermonQP, there are some concrete ones and also TAO wrapper. However, adding an Interior Point implementation is interesting for us as well. > > PermonSVM is so far a proof-of-concept thing, but it already scales pretty well (almost proportionally to the application of the data matrix to a vector). See, e.g. our PASC poster https://www.researchgate.net/publication/318317204_PERMON_PASC17_Poster > > We'll be grateful for any feedback on this. > > Jakub > > > On 22.9.2017 06:06, Richard Tran Mills wrote: >> Thanks for sharing this, Barry. I haven't had time to read their paper, but it looks worth a read. >> >> Hong, since many machine-learning or data-mining problems can be cast as linear algebra problems (several examples involving eigenproblems come to mind), I'm guessing that there must be several people using PETSc (with SLEPc, likely) in this this area, but I don't think I've come across any published examples. What have others seen? >> >> Most of the machine learning and data-mining papers I read seem employ sequential algorithms or, at most, algorithms targeted at on-node parallelism only. With available data sets getting as large and easily available as they are, I'm surprised that there isn't more focus on doing things with distributed parallelism. One of my cited papers is on a distributed parallel k-means implementation I worked on some years ago: we didn't do anything especially clever with it, but today it is still one of the *only* parallel clustering publications I've seen. >> >> I'd love to 1) hear about what other machine-learning or data-mining applications using PETSc that others have come across and 2) hear about applications in this area where people aren't using PETSc but it looks like they should! >> >> Cheers, >> Richard >> >> On Thu, Sep 21, 2017 at 12:51 PM, Zhang, Hong <hongzhang(a)anl.gov> wrote: >> Great news! According to their papers, MLSVM works only in serial. I am not sure what is stopping them using PETSc in parallel. >> >> Btw, are there any other cases that use PETSc for machine learning? >> >> Hong (Mr.) >> >> > On Sep 21, 2017, at 1:02 PM, Barry Smith <bsmith(a)mcs.anl.gov> wrote: >> > >> > >> > From: Ilya Safro isafro(a)g.clemson.edu >> > Date: September 17, 2017 >> > Subject: MLSVM 1.0, Multilevel Support Vector Machines >> > >> > We are pleased to announce the release of MLSVM 1.0, a library of fast >> > multilevel algorithms for training nonlinear support vector machine >> > models on large-scale datasets. The library is developed as an >> > extension of PETSc to support, among other applications, the analysis >> > of datasets in scientific computing. >> > >> > Highlights: >> > - The best quality/performance trade-off is achieved with algebraic >> > multigrid coarsening >> > - Tested on academic, industrial, and healthcare datasets >> > - Generates multiple models for each training >> > - Effective on imbalanced datasets >> > >> > Download MLSVM at https://github.com/esadr/mlsvm >> > >> > Corresponding paper: Sadrfaridpour, Razzaghi and Safro "Engineering >> > multilevel support vector machines", 2017, >> > https://arxiv.org/pdf/1707.07657.pdf >> > >> >> >

1 0

Re: [petsc-dev] (no subject)
by Jakub Kruzik 23 Sep '17

23 Sep '17

Dear all, I would just like to note that we also develop SVM implementation. It is intended for large-scale datasets and makes use of PETSc parallel linear algebra. Currently, it supports only linear kernels - Hessian is, in fact, MATNORMAL with arbitrary underlying data matrix - it is, e.g. possible to use MATDENSE or MATAIJ depending on the problem. For the solution of the arising quadratic program (QP), it uses solvers from our PermonQP package. Both PermonSVM and PermonQP are libraries depending on PETSc. They are written in the PETSc coding style, pretty much like SLEPc. http://permon.it4i.cz/permonqp.htm http://permon.it4i.cz/permonsvm.htm https://github.com/it4innovations/permon https://github.com/it4innovations/permonsvm So far, PermonQP only implements an Augmented Lagrangian type algorithm which can be combined with any solver for box-constrained QP. In PermonQP, there are some concrete ones and also TAO wrapper. However, adding an Interior Point implementation is interesting for us as well. PermonSVM is so far a proof-of-concept thing, but it already scales pretty well (almost proportionally to the application of the data matrix to a vector). See, e.g. our PASC poster https://www.researchgate.net/publication/318317204_PERMON_PASC17_Poster We'll be grateful for any feedback on this. Jakub On 22.9.2017 06:06, Richard Tran Mills wrote: > Thanks for sharing this, Barry. I haven't had time to read their > paper, but it looks worth a read. > > Hong, since many machine-learning or data-mining problems can be cast > as linear algebra problems (several examples involving eigenproblems > come to mind), I'm guessing that there must be several people using > PETSc (with SLEPc, likely) in this this area, but I don't think I've > come across any published examples. What have others seen? > > Most of the machine learning and data-mining papers I read seem employ > sequential algorithms or, at most, algorithms targeted at on-node > parallelism only. With available data sets getting as large and easily > available as they are, I'm surprised that there isn't more focus on > doing things with distributed parallelism. One of my cited papers is > on a distributed parallel k-means implementation I worked on some > years ago: we didn't do anything especially clever with it, but today > it is still one of the *only* parallel clustering publications I've seen. > > I'd love to 1) hear about what other machine-learning or data-mining > applications using PETSc that others have come across and 2) hear > about applications in this area where people aren't using PETSc but it > looks like they should! > > Cheers, > Richard > > On Thu, Sep 21, 2017 at 12:51 PM, Zhang, Hong <hongzhang(a)anl.gov > <mailto:[email protected]>> wrote: > > Great news! According to their papers, MLSVM works only in serial. > I am not sure what is stopping them using PETSc in parallel. > > Btw, are there any other cases that use PETSc for machine learning? > > Hong (Mr.) > > > On Sep 21, 2017, at 1:02 PM, Barry Smith <bsmith(a)mcs.anl.gov > <mailto:[email protected]>> wrote: > > > > > > From: Ilya Safro isafro(a)g.clemson.edu <mailto:[email protected]> > > Date: September 17, 2017 > > Subject: MLSVM 1.0, Multilevel Support Vector Machines > > > > We are pleased to announce the release of MLSVM 1.0, a library > of fast > > multilevel algorithms for training nonlinear support vector machine > > models on large-scale datasets. The library is developed as an > > extension of PETSc to support, among other applications, the > analysis > > of datasets in scientific computing. > > > > Highlights: > > - The best quality/performance trade-off is achieved with algebraic > > multigrid coarsening > > - Tested on academic, industrial, and healthcare datasets > > - Generates multiple models for each training > > - Effective on imbalanced datasets > > > > Download MLSVM at https://github.com/esadr/mlsvm > > > > Corresponding paper: Sadrfaridpour, Razzaghi and Safro "Engineering > > multilevel support vector machines", 2017, > > https://arxiv.org/pdf/1707.07657.pdf > <https://arxiv.org/pdf/1707.07657.pdf> > > > >

1 0