On Fri, Feb 3, 2017 at 11:44 AM, Matthew Knepley <[email protected]> wrote:

On Fri, Feb 3, 2017 at 11:41 AM, Barry Smith <[email protected]> wrote:

...

...

On Feb 3, 2017, at 12:59 AM, Mark McClure <[email protected]> wrote:

Hi, Jed and Matt.

To close the loop, it turns out that the condition number issue had a simple explanation. The BC equation that I had added to the system had very different units from the other equations. Multiplying that row by a constant (effectively, modifying the units of the equation) improved the condition number of the matrix by many orders of magnitude. Big picture, this did not have an apparent effect on any of the numerical performance - the linear solver still had nonconvergence in the same places (when I used a large number of processes) and when I use a smaller number of processors so that the linear solve always converges, the overall numerical scheme is unaffected by whether or not I scale the extra BC equation. I hadn't realized how important it is to make sure the equations are scaled consistently/nondimensionalized when using the convergence number to evaluate whether a system is singular. Interesting experience.

Thanks again for the help. I'll use the direct solver you suggested as a backup and send the user a warning if they try to use too many processors on a small problem.

An aside, at first, when I saw that my overall numerical scheme was failing, I didn't realize the problem was that the linear solver was not converging. I spent a fair amount time debugging until I found the issue and learned how to use KSPConvergedReason. Because if KSP doesn't converge, it merely returns incorrect numbers. Without knowing to run KSPConvergedReason, an inexperienced user (like me) many not know about the nonconvergence in the linear solve and could spend a lot of time checking for other issues. It might be worth changing the default behavior for nonconvergence to be that it returns an nan so that the user gets a clear signal that the values coming out of KSP cannot be used.

Because the linear solvers in PETSc are usually used by the nonlinear solvers and ODE integrators that have recovery methods for failure in a linear solve we moved away from the "crash and burn" on failed linear solver approach; since that makes the recovery more difficult.

We could consider the following; if the KSP is not created by a SNES or TS it defaults to "crash and burn" on failed linear solve this would help newbies who are only solving linear systems and expecting a "crash and burn" on failure.

I am for that.

I like this feature. Hong

Matt

...

What do people think?

Barry

...

Regards, Mark

On Thu, Feb 2, 2017 at 7:48 AM, Jed Brown <[email protected]> wrote: Mark McClure <[email protected]> writes:

...

I think you are right that I have an issue with how the BC is

implemented.

...

...

It is a pipe flow simulation that is solving mass and momentum balance simultaneously (corresponding unknowns are pressure and flow rate). I am applying a constant mass flow rate boundary condition. Upon further consideration, it may be that I am not properly providing a boundary condition for the momentum balance equation at the inlet. If I did, the inlet pressure could be readily calculated standalone,

Momentum inflow is common, but if you also have momentum outflow (i.e., all Dirichlet conditions for momentum) then there is a null space of constant pressure -- pressure is only determined up to a constant. See the user manual section on solving singular equations.

...

outside the system of equations, and the problematic equation would be removed.

-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener